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ABSTRACT . . xi. 

Tho studies were conducted tc deteriine tee 
educational effectiveness of innovative ccriespcndence course 
■ate rials. The studies yere aade siaultaneously during 
•from September 1S76 thrcugh October 1S77. Different 
compared in terms of student performance, student acceptance of 
materials, course completion time, and relative perfciHacce of 
students with differing reading abilities. In each study innovative 
textual materials were compared to materials alr.eady in field use. 
The first study involved a three-way comparison of conventional 
materials, texts in the behavioral objective format (EOF), 
innovative modular BOF version of an apprentice-material 
course. The second study compared a EOF version of a focd-service 
specialists course with a version using the innovative high impact 
graphic technigue. Findings fo|: the conventional material^ used in 
the first study were inconclusive, while in both studies each 
innovative concept proved to before effective and was tetter 
accepted than each respective BOF version. (Author/CSS) 
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ABSTRACT 



This paper describes and reports the findings for two studies con- 
ducted to determine the educational effectiveness of innovative corres- 
pondence course materials. The studies were made simultaneously during 
the period from September 1976 through October 1977. Different materials 
were compared in terms of student performance, student acceptance of 
materials, course completion time, and relative performance of students 
with differing reading abilities. In each study, _ innovative textual 
materials were compared to materials already in field use. 

Tfie first study involved a 'three way comparison of conventional 
materials, texts in the Behavorial Objective Format (BOF), and an 
innovative modular BOF version of an Apprentice Materiel Facilities 
course. The second study compares a BOF version of a Food Service 
Specialists course with a version using the innovative high impact 
graphic technique. 

* ■ Findings for the conventional ma/erials used in the first study 
were inconclusive while, in both studies, each innovative concept proved 
to bt more effective and was better accepted than each respective BOF 
version. , ' . % 

KEY WORDS: Behavioral objectives, correspondence courses, reading 
ability, textbook evaluation, textbook .illustrations, textbook research. 
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CHAPTZH I 



INTRODUCTION 

The Extension Course Institute (EC I) of the Air University was 
established in 1950 to provide consolidated ir^nagerient and adTiiinistra- 
tion of the United States .^ir Force correspondence education program. 
A major responsibility of the organization is to provide career develop 
nenr courses (CDC) ihat support the Air Force "dual char.nel" approach t( 
■e ""isted career progression. This concept supplements practical, super 
vir--^ on-the-job training (OJT) with complimentary specialty knowledge 
ch'^^.i^^h correspondence study. ^ The quality of these correspondence 
texts can have a major impact on job performance throughout the entire 
Air ^orce. 

Historically, few changes have occurred in ECI's correspondence 
naterials other than periodic currency revisions. Departures from the 
plain textbook appearance by innovations in physical format have been 
relatively minor. 

An exception to. the preceding statement centers around a 1973 
decision to convert all ECI textfe to the Behavioral Objective'^ Format 
(BOF). A description of this caaicept appears later in 'this chapter but 
simply stated, the method strives to insure minimum levels of achieve- 

■^Department of the Air Force, Qn-the-Job Trafning\ AF Manual 
50-23 (Washington: Government Printing Office, 197H), p. 1-1. 



ment through competency-based instructional techniques.^ At this writ- 
ing, about half of tCI's approximately 380 courses have been or are in 
the process of being converted to the BOF. Althougl;! studies of educa- 
tional innovations in the classroom setting are numerous, few empirical 
data are available to substantiate or refute the "educational merit of 
new concepts in the extension course environment. 

> vJovanovich (1969) is but one of many who emphasized the need for 

✓ 

flexibility and innovation in teaching materials in light of the tre- 
mendous technological advances being witnessed. Similar thinking led 
the Air University to initiate a pilot project in March, 197U to examine 

I 

the effectiveness of computer assisted instructional (CAI) techniques as 
a subistitute for correspondence study in the "dual channel" career 
advancement system. Two Career Development Courses were selected for 
Inclusion in this project; each to be programmed into an interactive 
computer /student medium, during subsequent field testing, students 
recfeiving training via these media w/uld comprise the experimental 
groups while r.cudents in the control groups were to receive instruction 
using hard, copy texts. These texti (described later in this chapter) 
were to be significant revisions of course materials already in field 
use and were to be produced using innovative formats. The utility of 
CAI techniques would then be experimentally assessed in terms including, 
but not limited to :. information currency, student interest, student 

— •■ 

■^W. Robert Houston and Allen R. Warner, "The Competency-Based 
Movement: Origins and Future," Educational Technology 17 (June 1977) :14. 

— — ^William Jovanovich, "The American Textbook: An Unscientific 
Phenomenon— Quality" Without Control," The American Scholar 38 (Spring 
1963): 234. 
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performance, and cont, 

In January 1976, the Air University Commander directed, a new look 

at the justification for proceeding with the CAI project it^ view of 

increasing fiscal pressures • The detailed reassessment by Christopher 

that followed resulted in a recommendation that stated: 

• ••in face of the uncertainty in th^ .near future of 
cost-effective CAI systogis suitable as an alternative 
for correspondence courses, ...the CAI dimension of 
the project should be terminated. ^ 

This recommendation was adopted in February 1976, and further ^ 

development of CAI programs ended. Alternately, it was recognized that 

since the innovative texts attendant to the CAI project were well under 

construction, they could be compared to existing traditional materials 

if their development was completed. ECI assumed the resp^sibility for 

thiSp comparison in March 1976 and developed an evaluation plan that 

cont-ained the -following specific purposes: 

a. To field test Career Development Course 62250, Food Service 
Specialist, presented in the Behavorial Objective Format versus a fortnat 

••containing contemporary narrative and high impact graphics. 

b. To field test Career Development Course 64531,- Apprentice 
Materiel Facilities Special^t, comparing three presentation media: 
conventional text. Behavioral Objective Format, and Modular Behavioral 



^A detailed description of lthis project is contained in "Document 
tation of the Extension Courses and Information .Systems Pilot Program 
Through 1 February 1976" (Headquarters, Air University, Maxwell AFB, 
Alabama).* ^ - 

^G. Ronald Christopher, "Review Analysis of the PLATO/TICCIT,.Test _ 
Project Conducted From 6 January 1976 - 15 January 1976"_(Headquarters , 
AirxUnivarsitys Maxwell AFB, Alabama), p. 22, • v 

\ 
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Objective Format. 

Descriptions of each of these textual treatments are covered later 
in this chapter with examples in the appendixes. 

Statennent of the Problem 

m 

The problem under consideration in this study was to gather data 
concerning the relative educational merits of differing textual format 
treatments. All data had to be gathered by mail, with no face-to- 
face institutional/student interaction, to enable statistically accur- 
ate comparisons of materials based on student achievement, student 
acceptance and motivation'^ accommodation of different reading abilities 



and time required for course completion. 



Significance of the Study 

The evolution of correspondence study in the Air Force from an 
offering of a few voluntary courses borrowed from the Army to the 
highly specialized, integrand and mandatory career development course 
system of today underscores the importance placed on the medixom by the 
service. It is the most cost effective method of providing, training to 
a student population numbering well over one-quarter million. If 
research can result in qualitative improvements, large payoffs in effi- 
ciency can be achieved • 

Much of the past research in correspondence education has centered 
around empirical studies designed to compare nonresident to resident 
training and to determine student reactions to correspondence study. 

Ossian MacKenzie, Edward L. Christensen, and Paul H. Rigby, 
Correspondence Instruction in the United States (New York: McGraw-Hill 



Book Co., IQee)-., p. 155, 



As alluded to earlier, little effort has been devoted to det^"-^ir ' the 

effects of textual format changes on educational quality. 

Mackenzie, in the Correspondence Education Research Projec v 

ernphas;.?:ed the distinct shortcoming in this area saying was no 

marked agreement, however, that a generally good job had bev one by 

most (correspondence) institutions on physical format."-^ Not^ i:-; this 

situation unique to correspondence materials/ McKeachie, referring to 

educational textbooks in general stated; "Despite the age of printing 

2 

as a technique, there is relatively little research on its use " 

The reasons for this deficiency, particularly in correspondence 
materials, are varied but seem to center on some basic realities. 
First, correspondence education is over 100 ysars old. This has caused 
the medium to become somewhat set in its ways thus generating a certain \ 
resistance to change. Secondly, the complexity and consequent high 
cost of innovating simply for^experimental purposes makes the corres- 
pondence medium a less attractive channel for research than other 
media. ^ 

Th^innovative texts produced for the CAI pilot project and ECI's 
firm committment to the need for continued and aggressive research to 
improve;its educational effectiveness provided the opportunity for a 



^Ibid., p. 179. 



^Wilbert J. McKeachie, "Research on Teaching at the College and 
University Level," Handbook, of Research on Teaching , American Educa- 
tional Research Association (Chicago: Rand McNally and Co.,. 1963), 
p. 1156. 

Leonard S. Stein, "Is Home Study a Stepchild?" Home Study Review , 
Winter 1961, p. 35. 
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contribution in' this deficient area, 

Hypot^V ' ^^es To Be Tested 

a. That students who re try instruction by way of modif ie : '^OCs 
(newly developed) will perfor::) Su.^nif ^cantly better on course exaii na- 
tions than those who receive instruction by conventiona^l texts. 

b. That high ability readers will perform significantly better 
than low ability readers on each respective field test course HMmina- 
tion« 

That, on the Food Services course examinations, high ability 

readers will show little difference in performance depending on training 

\ 

s 

media while low ability readers will perform significantly better using 
the contemporary narrative/enharjced graphics version • 

d. That, on the Materiel Facilities course examinations, nigh 
ability readers will show little difference in perfornnance depending on 
training ip.edia whereas low ability readers will demonstrate poorest 
performance using ccaventional texts, intermediate performance with BOF, 
and highest perfoi^ance u?. .,ig modular BOF texts. 

e. That students receiving modified t^t insti*uction will display 
a significantly more positive attitude towaijd their training than those 
who receive instruction by conventional CDCs. 

f* That the innovative texts will be completed in a sKprter aver- 
age time than current conventxonal and BOF volumes. 

Assumptions a'lid Limitations 
This study was primarily concerned with the academic (cnC7 pes^ 
formance of airmen participating in the dual channel training system. 
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Therefore, students who enrolled in the experimental courses voluntarily 
with no OJT cominitment were not included in the test population. Mem- 
bers of the Air Force Reserve and Air National Guard were included on an 
equal basis with their active duty counterparts in keeping with the 
total force concept as long as they we'?e mandatorily (OJT) enrol Led, 

It^^-^as recognized that OJT environments vary quali^tively between 
Air Force installations. Further, it was assumed that any data contam- 
ination attributed to this factor would be ii^nimized by the randoTh' 
assignment of subjects to treatment groups. These differences represent 
reality in the Air Force training prograi^ and thus categorized this 
Study as a field test as opposed to an experiment where Strict labora- 
tory controls could be achieved- 

Conducting a field test by tnail would result in a degrefe of data 
perishability. In addition to normal attrition in the test c »urses 
(10 - m percent in 1976), lost data would probably result from the 
necessity for voluntary participation in any survey procedure used.^ 
However, ^y enrolling at least 100 subjects in each treatment group, it 
was assumed that sufficient data would survive to enab4;e comparisons 
that were statistically sound. 

Finally, since nothing '^n past' ECI examination analyses indicated 
the contrary, it was assumed that minimal compromise existed in the 
currently used test instruments for each of the two study career fields. 



Department of. the Air J4)rce, Air Force Privacy Act Program , 

' ^ Printing Office, 1975) , 



AF ReguQ^ion 12-35 ( Washijp:gttn : Government 
p. 1. 
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fef init --f Terms 

^Specialty Training . : ^diard ( .TT' . ) "An STS outlines the training 
required to achieve a skJ.ll^'(^) leve^ ithin an airman Air Force Speci- 
alty (AFS). Through its use, the rsdividual training of airmen is 
standardized and the quality of training controlled. "-^ 

Conventional Text > The coinv^tional text is in straight ^narrative 
and, like, a standard textbook, coP'j^i^ins no explicit statement of objeo- 
, tives. The content of these texts is based solely on the Specialty 
Training Standard and the appropriate job description. (Appendix A.) 

Beha v ioral Objective Forma"^ (BOF) . Texts using the Behavioral 
Objective Format precede each JSlock of material with specifier objec- 
tives in behavioral terms that empfeaisize to the student desired „ :in 
outcomes./^ Following the material, exercises are provided that indicate 
to the student whether desired performance levels have been achieved. 
If not, the need for review is indicated. The BOF is the method cur- * 
rently used by preparing agencies for revision or creation of new 
courses. (Appeni^ix B. ) 

Modular BOF . The modular concept is similar to the standard ioF 
except that each chapter stands as its own building-block entity (hence 
the term module). The student may thus work these volumes in any chap^ 
ter order depending on his priorities. Other refinements inblude lim- 
ited numbers of highly sub j ect-f ocHsed illustrations and large spaces 
for marginal not^ns. (Appendix C) 

■^Department of the Air Force, Air Force Specialty Training 
Standards, AF Regulation 8-13 (Washington: (3ov"emment Printing Office, 
rS74), p. 1, J 



ConteTT^ o -^>iir' y Narrative/Enhanced Gri ^ phics . The contemporary nar- 
i^ative witn :'nnanced graphics concept uses profuse and varied illu5.tra- 
(tio^Si ch: vv „ and diagrams with a fictional character narrative to 
carry th« it' nded lesson. (Appendix D. ) 

CDC r^- rc-vi S: ervice Specialist) . This Career Development 
Course i J^raeci V) a^ssisT in the upgrading of Food Service personnel 

from be^, nnir;^:^ >r 'ent: evels to the semi-skilled level. The current 
course i in "S*. riehavicral Objective Format with t>)e experimental 
version produc-'i in contem^porary narrative/enhanced graphics, 

CDC ^^^-531 ( Apprentice .Materiel Facilities Specialist) . A Career ^ 
Development Course for entry level supply personnel. The current ver- 
sion of this course is in the conventional t^xt format with experimental 
texts produ^;ed in the Behavioral Objective Format and the modular BOF, 
A three way textual comparison was thus possible in this career field. 

Skill Levels , SkJ.ll levels are proficiency classifications used to 
upgrade enlisted personnel in each Air Force carreer specialty. They are 
numerically designated as follows: 1-level, helper; 3-level, semi- 
skilled (apprentice)*; $-level, skilled (Journeyman); 7-level, advanced 
(supervisor); and 9-level, superintendent. Materiel Facilities and Food 
Service^ sjificialistis participating in this evaluation were in formal 
upgrade training ~o the three and five levels respectively. 
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CHAPTER II 



REVIEW OF RELATED LITERATURE 



Lvi(ii253ice carefully done research into innovative textUtt fonnats 
is entrr^nne^:/ rr^re in educational literature,^ Even nnore absf^nt re 
sti-uiiiefi of ~"-is nature applied in the area of correiapondence emtcarzon. 
A look only a t j^latively few -years into the past provides xDSi^^ht : n!tr 
thin observatioi^. 

Wedermm ?r and Childs, in a 1961 assessment of t\ie achi^veTTcents arid 
r;ifDi:e£i:!:ials of correspondence study, explained the lov esteeir in i<hic:ti 
^niH^ m-edium was held by educators and the general public,^ Although raeany 
-^epuTable institutions existed (with their ef^ectiveiuess weU. dennon^^ 
strated), a few others served to create a falsely geirerali2e4 image ^kt^*" 
was certainly less than desirable. Claims such as "It's Easy' to Learm,'* 
"You, too. Can Enjoy a Big Salary," "Draw Me and Discover Your Talent," 
etc., had repeat edly^reinFcy»ced that single adverse image. ^ With cor- ' 
respondence study thus stigmatized by commercial appeals, it was under- 
standable that the national university structure ^as ireluctant to expend 
research effort in an area whose mode of operation it considered unpro- 
fessional and even unethical. Federal* Trade Commission interventiom 



r . ,. • . 

Charles A. Wedemeyer and Gayle B. Childs, New Perspectives ir. 
University Correspondence Educat ion. _CChlc^go : Center for the Study of 
Liberal Arts Education for AduJrts, 1961), ^ p. ^ 

2ibid., p. 71. ; - 



il 

served to remove the cause < -hi' cor" ^oondence stu<iv .•igma in the 
early sixties, but the mediun's recovt*- v into academ -edibility has 
been alow . ^ 

Wedemeyer and Childs defined som(^ jf the pressu hat had helped 

in sustaining this recovery. Modern demands for educ.„r-jon in increased 
variety, rapid social and tecrmai>r(i:,.-c.r^^ change, and c .v -*icr easingly 
mobile population had creat-sd ^vducatj^cnniaj. requi/ement:. ro the extent ^ 
that conventional means could not begin to cope with t*nG situation. 
Thus, correspondence study had, even by 1l961, grown inzo an important 
and integral part of th^^tiatioic's educatiional complex.^ 

The entire decade of the sixties saw continued growth of an inter- 
est in correspondence instruction. This national attention led to a 

study by Mackenzie, Christensen^ anc Rigby which constituted the most 

I ' 

thorough and searching examination of the medium to date. This Corre- 
spondence Education Research Project (CERP), published^in book form in 
1968, was stimulated by the National Home Study Council and the National 
University Extension Association under a grant from the Carnegie Cor- 
poration cf New York. The study covered the historical development of 
the medium ard examined programs offered by a variety of institutions; 
universities and colleges, governments, religious organizations, . the 
Armed Forces, industry, and private home study «?chools. It examined 

9 

problems facing correspondence study such as financing, acceptance by 
Students and educators, and accr^editation, and analyzed the effective- 



^Ibid. , p. 5, 
^Ibid. , p. 3, 
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nesn of the merhod as an educational vehicle ^ both pref^and con. Finally-, 
the study described future requireinents necessary to sustain the integ^'a- 
tion of the correspondence medium into the national educational structure. 
Of particular impprtance to the prese^pt study was the emphasiis on the 
general lack of attention given to the physical formats of textual , 
materialfe.-^ Thus, the need for research in this area was .again suggested* 

In 1971, Childs surveyed the current status of research in cor- 
respondence education. Although she noted a recent increase in Research ♦ 
she stated "...this does not indicate any great upsurge in research 
activity. Evidences of carefully done research are still hard to find-"^ 
The studies she described were categorised according to reactions of ^ 
students, studen^ charapt eristics, completion rates, methodology, and 
achievement. Seventeen studies were presented in addition to the CErP 
with only one addressing methodology. Even that one did not treat 
textual format. Again, the deficiency m this area of research was 
implicit. *s. ^ 

As described in the preceeding chapter, ECI began convorting its 
textual materials to the Behavioral Objective Format in 1973. In that ^ 
year, Duchastel and Merrill published a review of twenty^eight studies 



^MacKen^ie, Christensen, and Rigb;^', Correspondence Education in 
The United States , p. 179. 

^Gayle B.^ Childs, "Recent Research Developments in Correspondence 
Education" in The Changing World of Correspondence Study , eds. Ossian 
MacKenzie and Edward Christensen (University Park: The Pennsylvania 
State University Press, 1971), p. 229. 

^Ibid., pp. 236-37. 

/ 
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each of which tried to evaluate various aspects of the behavioral 
concept. These studies were categorized depending upon the specific 

evaluation objective: effects on learning (achievemetit ) , type of learn- 

j 

ing involved (knowledge versus comprehension), learner characteristics 
and time to criterion. In ten studies evaluating achievement, no con- 
i;istent benefit or degradation wal observed among students exposed to 
behavioral object ives,^ Of seven studies concerned v/ith types of learn- 
ing, only one found objectives beneficial (in the knowledge area). 
Findings in the six other studies were inconsistent,^ Eight studies 
were concerned with learner characteristics. Of these, one found that 
^behavioral objectives benefited mid-ability students who were grouped 
based on Scores in previous courses and one claimed that behavioral ^ 
objectives reduced levels of state anxiety.^ The remaining s5x were 
inconclusive when students were stratif ied on apliitude and standard 

ability tests. Three studies analyzing time to criterion were incon- 
u 

sistent* 

In 197a, ECt completed an internal study designed to compare the 
relative effectiveness of correspondence materials written in the BOF as 
opposed to those written in a conventional textbook manner. The results 

■ - ' \ 

of that study indicate<^ little or 'no difference between treatmer ' groups 



■^Philippe C- Duchastel and Paul F. Merrill, "The Effects of Behav- 
ioral Objectives on Learning: A Review of Empirical Studies," Review o^ 
Educational Research U3 (Winter 1973): 57. 



^^Ibid. , p. 59. 
^Ibid. , p. 61. 
- Ibid*, p. 62. 



24 



with students using different textual formats. However, th^^re was some 
statistically non-significant evidence that the BOF offered some benefit 
for lower ability 'students , 

This st\jdy was subject to several limitations. Chief among these 
was the fact that the c;ubject population was working toward the 7-level 
(supervisor) in the test career field,- The primary difference in re- 
quired knowledge between^he 5-level (semi-skilled) and the 7-level is 
therefore in the supervisory area with most technical knowled^^t already 
acquired by the 5-level. It could therefore be concluded that some 
(possibly many) subjects already possessed sufficient technical infor- 
mation to successfully pass the 7-level end-of-course examination with- 
out additional study. 

Secomlly, the 6nly criterion variables used were achievement 
scores on a pretest and later post test. Little consideration was , 
given to uncontrolled variables which could have influenced test results. 
Siegel, Lautman, and Burkett, for example, found a significant improve- 
ment in criterion test scores among subjects using home vstudy materials 
with a lowered reading grade level. ^ A reasonable extension of thi^' 
finding is that if textual readability were held constant, individual 
differences in student reading ability would haveV^ predictive value in 
terms of test performance. in a 1974 position statement, the School* 

"^ChaMes L. .McMillin, "Behavioral Objectives Format Field Ser- 
vice Test: Career Development Course 30474X" (Extension Course' Insti- 
tute, Gunter AFS, Alabama, 1973), p. 5. 

2 

Arthur I. Siegel, Martin R. Lautman, and James R. Burkett, 
"Reading Grade Level Adjustment and Auditory^Supplementation as Tech- 
niques for Increasing Textual Comprehensibirity ," Journal of Educational 
Psychology 66 (December 1974):901. 



\ 

Divir»ion of the Association of American Pu!)lishers noted that it cannot 
I 

be sugge-sted that materials alone' the sole determinant of student 

performance. Many other variables should be considered such ar the 

nature of the students themselves, their attitudes, their abilities, and 

.he environmental influences in the learning situation.^ Only by so 

doing can the accuracy of textual assessments be increased. 

The lack of comprehensive textual research is not unique to the 

\ 

correspondence study environment. Brodinsky, in 1975, reported on the 
state of 'the educational textbac^k art. The industry is larger , therefore 
changes to meet the demands of educators come slowly. Response has 
tended to reflect, current pressures — elimination of bias and discrim- 
ination, increased emphasis on vocationally oridnted materials, and even 
metric conversion have become first line concerns which have forced 
innovation into secondary importance^ "Learner Verification" or asses- 
sing the effectiveness of new materials has been defined as the indus- 
try's most pressing need.^ Publishers have acknowledged this need and 
have begun work on the problem. However, they admit that they have not 
yet formallized procedures for reporting their findings to educators.^ 

In summary, the literature contains littfe in the area of resiearch 
into innovative textual formats, though -it suggests the plausable ^ 

-^Lawrence Lipsitz, ed. , "News Notes," Educational Technology ^ 
m (March 1974):63-6U. ■ • . 

Ben Brodinsky, "Instructional Materials: The Changing Industry," 
NASSP Bulletin 59 (October 1975): 54. . 

'^Ibid. , p. 56. ■ • ' * ' 

* 
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reasons for this deficiency»^ More importantly, in identifying needed 
research, the literature suggests considerations and directions that 
have influenced the design of the ciarrent st:udy« This design is disciissed 
in chapter III. 



Shipp, whose doctoral thesis was not available until this report 
was in final draft, reached a similar conclusion about the lack of clear- 
cut findings in the literature. See William Lee Shipp, Evaluation of the 
Behavioral Objective Format As an Effective System for Air Force Corres- 
pondence Courses , University of Illinois at Urbana-Charhpaignt (anticip.^ted) 
1978, Though he noted equivocal findings, Shipp' s study showed the Be- 
havioral. Objective Format to be superior to conventional tekts as measured 
by exam scores (12.8% increase) and completion times (26% decrease). 
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CHAPTER III 



f 



DESIGN OF THE STUDY 



Description of Research Design 



Although the statistical methods used to evaluate each CDC were 
similar, this study was essentially two separate field tests. This dual 
approach was necessitated by the differing nature of the two CDCs origi- 
nally selected for the Computer Assisted Instruction evaluation. Each 
course currently used in the field deals with different subject matter 
and is in different formats, i.e., conventional textbook and the BOF. 
The modified (field test) versions of each were also prepared in dif:^r- 
ent formats, i.e., modular BOF and contemporary narrative/enhanced 
grc-:phics. Also, since each CDC was oriented toward different skill 
levels, no cominon basis existed upon which to build measured comparisons 
between them. Thus, the only viable approach was to evaluate each CDC 
as a separate study. Figure 1 graphically illustrates the test career 
fields and the textual materials available for each. * 

Students enrolled in the currently used course versions constituted 
the control groups while those enrolled in each innovative version were 
experimental. A variation from the traditional, experimental and control 



group concept occurred with the Materiel Facilities Specialists since 
two innovative texts were available'^ for study. Subjects u^ing these two 
sets of materials were compared not only to those in the control group 
but also to each other. 




IB 



FIGURE I 

:0URSES AVAILABLE FOR EVALUATION 



Food Service 
(CDC 62250) 



Materiel Facilities 
(CDC 6U531) 



Behavioral Objective 
Format 5^ 



Contemporary Narrative, 
Enhanced Graphics 



Conventional Text" 



Behavioral Objective. 
Format 



Modular BOF 



CDC versions resently in field use. 

''""^ The Materiel Facilities CDC was converted by ECI to the BOF 
specifically for this field test. Thus, a comparison between 
three textual treatments was possible. It was not considered 
feasible to generate a conventional text for the Food Service 
specialty. 

In each career field, data were gathered that enabled between group 
comparisons of dependent variables consisting of criterion test scores 
(achievement), course completion time (in days), and relative student 
acceptance of materials. In addition to differing textual formats, a 
measure of student reading comprehension was used as an independent 
variable both for treatment group assignment and for i;ithin group 
strat if icatioji. The specific data^^gathering instrioment used and its 
applications are discussed later in this chapter. ^ 

Sampling Procedures 
Data collection within the Food Service career field began prior to 
the actual field test of experimental materials. Since approximately 
eighty-five percent of ^^^"^ Service personnel receive their initial 
training (3-level) at a resident school, an opportunity existed to 



2S 



gather preliminary information on potential subjects for the cur^^ent 
study. The resident Food Service faculty at Lowry Air Force Base, 
'oloraio, administered to 210 resident 3-level students a test battery 
thdt contained a pretest of 5-level material, a measure of reading 
compr - hens ion, an assessment of student attitudes toward tiheir career 
fieli, and a limited survey of demographic data. (Appendix ) 



As students in each test career field enrolled in their respective 
CDCs^ enrollments were screened to determine field test participation 
eligibility based on mandatory (OJT) status. Those Food Service per- 
sonnel fof^ whom preliminary data were available were assigned to control 
or experimental groups based on reading comprehension scores and sex. 
Thin prestratif led e^.ch trtjatment group with a proportionate distri- 
bution of high and low ability subjects. The remaining Food Service and 
all Materiel Facilities students were assigned to treatment groups using 
a computer generated random ^sequeijce to minimize systematic bias and the 
effect of extraneous variables. Enrollment applications were then 
processed through the normal ECI automated system and appropriate mate- 
rials issued. 

Unique items enclosed with each set c|f materials included a post 
card form on which the student was requested to indicate the date of 
course material receipt and a set of instructions for the student's OJT 
monitor (see appendixes F and G). These instructions simply acknow^ 
ledged the existence ot a study in progress and enlisted the training 
monitor's support in insuring that each student used only the materials 



provided. 
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enrollment goals were sftt at a minimum of 100 subjects in each 
treatment group in each career freld. Prior experience with the enroll- 
ment characteristics (and anticipated attrition) of the test courses 
indicated that sufficient numbers would reach course completion 
to assure a statistically adequate population.^ 

After each student finished his pixJgramXof study, an end-of -course 
examination (CE) was sent in accordance with normal ECI procedures. 
Accompanying this instrument was a test and survey battery designed to 
obtain necessary field test data* The battery included an attitude 
measurement device that assessed like or dislike for the course mate- 
rials studied, an opinion survey and, for^ Materiel Facilities students 
and those Food Service specialists for whom pretest data were not avail- 
able, a reading comprehension measurement and request for demographic 
data. Examination administrators were then instructed to return the CE 
answer sheets and all survey data to ECI in specially identified 
envelopes. » 

A detailed profile of student characteristics in each treatment 
group for each test course is shown in appendixes H and I , 

Data Gather i^ Instruments 
For the purposes of this study, course completion time was defined 
as the number of days between course package receipt and the date each 
subject received his end-of-course examination. A postcard was sent 
with each set of materials on which the student could enter the date of 



"^Gene V. Glass and Julian C, Stanley, Statistical Methods in 
Education and Psychology (Englewood Cli'.ffs: Prentice-Hall, Inc., 
19^0), pp. 376-77. 

o > 31 . ■ 
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receipt. A space was provided on each answer sheet for the date of 
examination accomplishment. A comparison of these dates yielded a net 
time (independent of mail transit time) which was averaged for compari- 
sons between groups. 

A somewhat ambitious but* more refined measure of completion time 
was attempted by asking each student to record the amount of cJ.ock time 
spent during each study session on a forro provided. It was recognized 
at the outset that placing a responsibility of this nature on the 
student might be unreliable buti with large'^tudent groups, might prove 
an indicator of gross differences. The return of these forms was so low 
(less than 4 percent) and the entries so diverse that consideration wai§ 
meaningless. 

Student performance 

Individual student perfonnance measures were extracted from each 

end-of-course examination. Since, in both career fieldi^, materials and 

tests differed slightly in objective coverage, subscores were derived 

that included only knowledge assessment common to all groups being 

compared. 
J 

Reading comprehension 

To assess reading comprehension, a Cloze test was administered. 
Each student was given selected passages to read that related directly 
to his career field in which approximately every fifth word was omitted. 
The reader was required to supply the exact missing word by relying on 
his knowledge of the material and the dontext of the passage. The 
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CiOjZe instrument is administered in accordance with self-contained 
instructions and proved to be easily un^rstood by students. According 
to research, this type of test is reliable and has demonstrated con- 
struct validity.-^ (Gee example at appendix J,) 

Attitude measurement 

Student attitudes were assessed using the semantic differential 
technique (example at appendix K), Subjects were requested to choose, 
on a seven point scale, between a series of bipolar adjectives such as 
"good - bad," "weak - strong," etc, in response to a specific question. 
The question asked of resident Food Service subjects was "I feel the 
Food Service specialty is" and, at course completion, all subjects in 
both career fields were asked "I feel that my ECI textbook materials 
were." Student responses were^ averaged to provide means for comparisons 
between groups. Research has shown the reliability of this technique, 
which is easily administered by untrained persons, to range froir^0,88 to 

Questionnaire 



A twenty^f ive item questionnaire was also administered after 



ination completion (example at appendix N). It prgrt^^d a means for 
gaining answers to specific questions pertinent to this ii^vestigatibn. 



exam^ 



^Raymond J, HortOn, "The Construct Validity of Cloze* Procedure: An 
Exploratory Factor Analysis of Cloze, Paragraph Reading, and Structure- 
of- Intellect" (Ph.D. dissertation, Hofstra University, 1973), p. 113, 
CUniversity Mi^ofilms No, 73-30,000.) 

2 

Arthur Mathis, Timothy A. Smith, and Duncan H, Hansen, ''College 
Student's Attitudes Toward CAI," Journal of Educational P sychology > vol. 
61, no, 1 (1970), p, 50. [ 
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'^h(^ in'ni'unent was constructed from a bank. of standard LCI survey 
Questions and carried USAF r.urvey Control Nxmber 76-lUl. Percentages 
^or each response were used for analysis. 

Statistical Treatment 

The primary vehicle for data storage and manipulation was the Air 

^t^i^'^ersity Educational Time Sharing System. Data files were created 

"^ha*^ allowed within group calculations of standard statistical values: 

ni^an> median, mode, variance, and standard deviation. Group mean values 
» 

^er® computed for performance scores, reading comprehension, semantic 
^Uf^erential scores, and time for course completion. In both field test 
c^r^^r fields, analysis of variance techniques were used for' compairsons 
fcietween groups. Additionally, a ^treatment by level design was used to 
'tes'^ for interaction effect between performance scores among students 
^it^ differing reading abilities. In all cases, statistical signif- 
icance at the .05 confidence level was used as the criterion for hypo- 
"^he^is acceptance. Each Career Development Course is analyzed separ- 
^tely in the next two chapters, and specific data analysis procedures 
Used are discussed. 



J 
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CHAPTER IV 
MATERIEL FACILITIES SPECIALIST STUDY 
Introduction 

In this chapter, the Materiel Facilities Specialist study will be 
described. First the student population and its assignment to experi- 
mental groups is defined. Then^ the measures used for test in. hy^jo^ 'aeses 
are discussed and statistics for the measures are given, 'vext, the 
means of forming high and low reading ability groups a^e discussed, and 
f 'nally, a (description of, and report on each hypothesis tested is given. 
T:.is ils followed by a summary of the hypotheses tested. 

The Student Population and its Assignment 
to Experimental Groups 

The original subjects in this, study were 302 students who enrolled 

in the Materiel Facilities Specialist course between 13 October 1976 

and 5 January 1977. Subjects were randomly assigned to one of three 

I 

groups, for which different course materials had been developed. Group 
one received conventional materials, group- two received materials in the 
Behavioral Objective Format (BGF), and group three received modular EOF 
materials. 

The subjects were randomly assigned to treatment groups as they 
enrolled in the three-level Materiel Facilities apprentice-level course 
'(CDC 64531). Originally, 101 subjects were assigned to group one 

r 

r 
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( convent icnal), 100 to group two (30F), and 101 to group three (modu- 
lar). .Of the original 302 subjects, sufficient data were avail^le on 
233 at the close of the study. Group one (conventional) and groups two 
(BOD retained 75 subjects each and group three (modular) retained €3 
f^ubjects. This ainounted to a loss of 59 subjects, or 23 percent of the 
original population. As forecast in chapter I, about 10 percent of the 
population was lost through no-rr.al attrition. The rerr.. "nder were lost 
due to incomplete data. Sim, ''i part -ic. P.vt ion in the sr:i:dy was on a 
voluntary basis, losses beyorid normal .irtrition were .inricipated. A 
summary of population assignmient and L-.':;iSes is in Tab_ 1. 



TABLE . 

POPULATION^ ASSIGNMOnS AND LOSSES 



Type of Course 
Material 


Conventional 


BOF 


Modular 


Total 


Original number 
of subjects 


101 


100 


101 


302 


Subjects lost 
by attrition 


13 


13 


7 


33 


Subjects remaining 
after attrition 


88 


87 




269 


Subjects lost due 
to incomplete data 


13 


12 


11 


36 


Final number 
of subjects 


75 


75 


83 


233 



In appendix H, distributions showing demographics by group assign 
ment, including race,, sex, education, marital status, and number of 
dependents are given. The distributions reported are what would be 
expected through random assignment of subjects to each group. 
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Measures Used For Testing Hypotheses 
•cur r.easures were used to test the hypotheses; the statistics for 
e-iTh '^rr'^-rr, in 'Table 2- 

a. The ccr.vent ionai , BCF, and ncdular course rr^aterials differed 
:r. or er^ ivei:: rcvrred as well as the ways of presenting surject matter. 
:r.e ccur^-e exa'^iin^.t i on (CE) for the con^'ent ional course had 82 items, 
the BCF 75, and the rri'^^ular 93. A total of U7 items were the same on 
all three examinations and were used as a corr.oarable measure of group 
performance . 

L. The sematic differential technique was used to obtain a 

measure of each subject's attitude toward his course materials. The ^ 

J 

attitude scale ranged from minus 3 to plus 3 with zero being the po^nt y 
of indifference. 

c. A Cloze test was constructed from typical Materiel Faci- 
lities text materials and administered to each subject. These scores 
were used as a measure of reading coniprehension. 

d, A measure of completion time was obtained by subtracting 
the date on which each student received his course materials from the 
date that the student completed the course examination. 

A similar population in each group is indicated by the similarity 
of the means and medians for each measure; the standard deviations also 
indicate similar distributions. The measures used showed nc indication 
of having violated the basic assumption pf normality that Underlies 
statistical treatments used in this study (primarily the F-test, as used 
in analysis of variance techniques). 




TABLE 2 







lb i iLb o 


K MEASURES USED 


THE STUDY 
















Standard 






Score 


N 


Mean 


Median 


Mode 


.Deviation 


Range 


Reliability 


Ferfomance . 
















Convent i oTia 1 


n z 


/U. 31 


76 


72 


9. 54 


40-31 




Hor 


7 5 


65.99 


65 


63 


9.92 


44-91 




Modular 


75 


69.95 


68 


65 


11.16 


46-93 




Total 


^ J J 


/U. 0 / 


72 


65 


10.76 


40-93 


.72 


oenian L xc 
















uirreren L lax 
















Conventional - 


75 


32.4 


33 


32 


6.53 


8-49 




Bor 


75 


30.4 


30 


35 


6.57 


13-4b 




Modular 


83 




32 


33 


O . D / 






Total 


233 


Jl. b 


37 


35 


7.35 


3-53 


NA 


Cloze 
















Conventional 


75 


■ 13.32 


14 


17 


7.07 


1-27 




Bor 


75 


14.02 


15 


15 


8.60 


0-30 




M-ii;lar 


83 


15.37 


16 


23 


8.94 


0-36 




Total 


233 


14.28 


15 


22 


8.28 


0-36 


.85 


Completion Time 














Conventional 


49 


66.27 


64 


48 


23.33 


13-146 




Bor 


U9 


65.31 


61 


L9 


28.41 


28-205 




Modular 


55 


72.16 


69 


69 


23.35 


27-154 




Total 


149 


68.15 


65 


64/69 


25.26 


13-205 


NA 



The reliability coefficients for the performance and Cloze scores 
were calculated by the Kuder - Richardson 20 formula for internal con- 
sistence of responses.-^ The performance score reliability (.72) had a 
standard error of 2.72 which would result in an individual score variation 
at the 99 percent confidence level of 7.01. The Cloze score reliability 
(.85): had a standard error of 2,79 resulting in an individual score 



1^ 



George A. Ferguson, . §,tatistical Analysis in t^sychology and Educa- 
tion (New York: McGraw-Hill Book Co., 1971), p. 368. 
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variation at the 99 percent confidence level of ^ 1.21. Reliability 
coefficients for the semantic differential instrument and coinpietion 
times were not calculated. 

The Formation of High and Low 
Reading Ability Sxibgroups 

Two of the hypotheses required the formation of subgroups consist- 
ing of students who demonstrated different levels of reading ability. 
Two subgi^oups were formed, a high reading ability sxibgroup and a low. • 

The following procedure was used in forming subgroups. First, it 
was noted (see Table 2) that the mean for the Cloze score for all treat- 
ment groups combined was m.28. At the 99% confidence level an in- 
dividual Cloze score would vary no more than plus or minus 7.01. 
Considering this, i*t was decided to use the mean as a center point and 
separate the high and low reading ability groups by at least 7 points. 
Thus the high reading ability group included all students who scared 18 
or above on the Cloze test and the low reading ability group was com- 
posed of those that scored 10 or below. The frequency distribution of 
the Cloze scores for the high and low reading ability groups has been 
tabulated in Tables 3 and 



TABLE 3 



FRIQUENCY DISTKIBUTION OF CLOZE SCORES 
HIGH READING ABILITY SUBGROUPS 











Matched 




Conventional 


BOF 


Modular 


Clcze Scores 


Score 


Frequency 




Frequency 


Frequency 


36 






1 




35 










3^* 










33 






1 




32 










31 






2 




30 




2 


1 








2 


1 




28 




1 


X 




27 


1 




2 




26 




2 


2 




25 


- 1 


2- 


3 


1 


2U 


3 


1 


2 


1 


23 


2 


2 


5 




22 


6 


5 


3 


3 


21 


3 


3 


3 


•3 


20 


1 


u 


3 


J. 


19 


2 


3 


2 


2 


18 


U 


2 . 


u 


2 






TABLE 








FREQUENCY DISTRIBUTION 


OF CLOZE/SCORES 






LOW READING ABILITY SUBGROUPS 


• 










Matched 


Score 


Conventional 


BOF 


Modular 


Cloze Scores 


10 


2 






2 


9 


2 




3 


2 


8 


3 




1 




7 


5 








6 


2 




2 


2 


5 


U 


S 


2 


' -2 


4 


2 




2 


2 


3 




1 


3 


1 


2 


3 - ^ 


■ B • 


3 


3 


1 




*i 


3 


1 


0 . ^ 




i 


1 





40 ^ 



30 



Hypotheses Testi3(j 



In all, eleven hypotheses were formulated and tested for the 



Materiel Facilities portion of the study. A description of each hy- 



potheses and the resulting findings will be given in this section. The 
following format will be used for describing each hypothesis: First, 
the hypothesis will be stated in its experimental form. Next, the 
statistical techniques used for testing the hypothesis will be cited and 
the findings will be reported. For each experimental hypothesis tested, 
significance at or beyond the .05 confidence level was used for accept- 
ing the hypothesis as true. Analysis of variance techniques used 
followed Ferguson's descriptions of one and two way classifications.^ 

The first three hypotheses were grouped together as hypothesis "a" 
in the original proposal. As the first three hypotheses all perta'in to 
treatment effects en students' performance and use the same statistical 
technique, they will be described together. 

Effects of instruction on performance 

Hypothesis 1 was that students who study modular course materials 
would show significantly better performance scores than those students 
who study BOF materials. 

Hypothesis 2 was that students who study modular courses would show 
significantly better performance scores than those who study convention- 
al materials. 

Hypothesis 3 was that students who study BOF materials would show 



^Ibid. 



» P- 



208-45. 



m 




r, ignif icant iy better performance scores than studentn who study conven- 
tLonal materials. ^ 

The prior expectations were that examination performance by stu- 
dents would be greatest for group 3 (mo'dular), less for group 2 (BOF), 
and least for group 1 (conventional). As noted in Table 2, the actual 
performance means differ froiTi the expected rank order. The actual rank 
order was conventional (mean = 74.31) highest, modular (mean = 69.95) 
less, and BOP (mean = 65,99) least, 

A one way classification analysis of variance was used to test each 
hypothesis. A comparison was made between performance scores for modular 
^nd BOF students to test the first hypothesis. The comparisons indicated 
a significant difference (F(l,156) = 5^52, P < .05) favoring the modular 
students* performances; thus, the first experimental hypothesis was 
accepted as being true. 

A comparison was made between performance scores for modular and 
conventional students to test the second hypothesis. The comparisons 
indicated a significant difference, (F(l,156) = 6.87, P < .05) favoring 
the conventional student's performance. ThisS*.^ a significant but 
opposite finding of what was expected and consequently, the second 
experimental hypothesis was rejected. 

A comparison was made between performance scores for BOF and con- 
ventional -tudents to test the third hypothesis. The comparison indi- 
cated a highly significant difference, (F(l,li48) = 27.39, P < .001) 
favoring the conventional students. This, too, was a significant but 
opposite finding of what was expected, therefore, the third experimental 
hypothesis was rejected. 



3? 

Thu? ranornalies described in the preceding two paragraphs appear to 
be due to differences in voluirje review exercise and course examination 
overlap between the conventional and both BOF course versions. This was 
suggested by student responses on the opinion survey and is treated more 
fully in chapter VI. 

\ 

Effects of reading ability on performance J 

The fourth hypothesis was that high ability^ readers will perform 
significantly better than low ability readers. To test this, a compari- 
son was made between performance scores of 88 high ability reciders 
(students who scored above 17 on the Cloze test) and 85 low ability 
readers (students who scored below 11 on the Cloze test). The compari- 
son indicated a highly significant difference (F(l,171) = 13.73, P < ' 
.01) favoring the high ability readers. Thus, the fourth experimental 
hypothesis was accepted as being true. 

Unique effects of treatments on reading ability levels 

The fifth hypothesis was that high ability readers would show 
little difference in performance from one. treatment to another while low 
ability readers would demonstrate poorest performance using conventional 
texts, intermediate performance with BOF, and highest performance using 
the modulc.r materials. 

Hi|h and low reading ability groups were formed using a procedure 
similar to that of the fourth hypothesis. The assignment of subjects 
differed^ however, in that it required the matching of students in each 
of the groups based on their Cloze test scores. The number of matched 
groups for each score are shown in Table 3. The range of Cloze 



43 



33 

scores for the high matched groups was from 25-18 and the range for the 
low matched groups was from 10-1. 

As an example of the matching process, note that in Table 3, for a 
Cloze score of 22, the conventional treatment had six subjects, the BOF 
five, and the modular, three. Thus, for Cloze score 22, a maximum of 
three matched subjects was set by the modular group and subjects from 
the other two groups were randomly selected for assignment as matched 
subi v^cts , 

Table 5 shows tncr number of students for each cell, their mean CE 
scores, and the standard deviations of these scores. 

TABLE 5 

GROUP STATISTICS BY READING ABILITY LEVELS 

Reading 
Ability 

Levels ' Conventional BOF Modular 



Number of Subjects 15 15 * 15 

HIGH Mean CE Score 79,00 69, UO 73. 

CE Standard Deviation 7,46 7.3H 8.91 

Number jof Subjects 15 15 15 
LOW \ Mean CE Score 73.20 63,87 SQ.'^d 
.^'"^ CE Standard Deviation 9,20 8, 38 8,03 

The fifth hypothesis was tested by a two-way classification analy- 
sis of variance, computed on matched students assigned to each treatment 
at two levels, in a treatment by levels fixed model design, A compari- 
son was made of the variability of the differences between the cell 
performance score means at each level across treatments. The average 
scores reading ability and text format are shown in figure 2. The 
two-way interaction between treatments (conventional, BOF, and modular) , 
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and levels (high and low reading ability) was not significant (F(2,84) = 
.02). Thus, the fifth experimental hypothesis was rejected. 

' FIGURE 2 
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J^J^r, 73.47 



73.00 ^^-^ -Readet^^ 



68.53 



63.87 



Conventional • BOF Modular 

The course examination perf or :e curves for high and low 
readers are essentially ^ -ileJ . This illustrates a lack 
of interaction effect w again causes rejection of the 

hypothesis. 

Effects of format on student attitude 

The next three hypotheses (6, 7, and 8) pertain to the students' 
attitude toward their courses. (These three hypotheses were derived 
from hypothesis "e" of the original proposal.) As the three are related 
and use the same statistical technique, they will be described together. 

The sixth hypothesiL> s that students, who studied the modular 
course would display a significantly more positive attitude toward 
their training materials than those who receive the BOF version. 

The seventh hypothesis was t^at students who studied the modular 
course would display a significantly more positive attitude toward their 
training materials than those who received the conventional course^ 
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The eighth hypothesis was that students who' studied BOF course 
materials would display a significantly more positive attitude toward 
their course materials than those who received the conventional course. 

The prior expectations were that the modular course materials would 
be most favorably received, the BOF materials less, and conventional • 
materials least favorably received. As noted in Table 2, the actual 
semantic differential score means differ from their rexpected rank 
order. The actual rank order was conventional, 32.4, most favorably 
received; modular, 31.9, less favorably received; and BOF, 30.4, least 
favorably received. The scores are shown in Figure 3 in their oririn- 
ally scaled forms ranging from minus 3 to plus 3. 

FIGURE 3 

DISLIKE "? 9- ^^^^ 

I Conventional ^24 

1 Modular .20 ^ 

^BOF .04 

Each hypothesis was tested. using a one-way analysis of variance. A 
comparison was made between semantic differential scores for modular and 
BOF students to test the sixth hypothesis. This comparison indicat^vi 
modular students, more -than BOF students, viewed their course materials 
favorably; however, the difference, (f(1,156) = 1.71) in the semantic 
differential scores for the two was not significant. Thus, the sixth 
experimental hypothesis was rejected. 

The seventh hypothesis was tested by making a comparison between 
semantic differential scores for modular and conventional students. 
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This comparison indicated a reversal of wfaat was expected in that con- 
ventional students, more so than modular students, viewed their course 
materials favorably, however, the difference, (F(l,156) = .12) was not 
significant. ^ Hence, the seventh hypothesis was rejected. 

A comparison was made between semantic differential scores for BOF 
and conventional students to test the eighth hypothesis^ The comparison 
indicated a reversal ofwhat was expected in that conventional students 
more so than BOF students held favorable views of their course materials 
and consequently, the eighth experimental hypothesis was rejected. The 
unexpected finding was not significant, (F(l,m8) = 3.63, P < .10). 

Effect of format on completion times 

The ninth, tenth, and eleventh hypotheses pertained to completion 
time and wer^e formulated as restatements of hypothesis "f" of the 
original 'proposal. As they are similar they vdll be described together. 

The ninth hypothesis was that the completion tjone^^jir students 
using modular materials would be significantly shorter than for students 
using BOF materials. 

The tenth hypothesis was that the completion time for students 
using modular materials would be significantly shorter than for students 
using the conventional materials. 

The eleventh hypothesis was that the completion time for students 
using BOF materials would be significantly shorter than foip students 
using conventional materials. 

The prior expectations were that students who received Instruction 
by way of the modular course materials would require the least amount of 
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trim*^ to complete their course, students who received instruction by way 

r 

of- Ib^ HOP matericTrn would require more, and those who used the conven- 
ti^^nal materials would require the most time. The actual rank order for 
Qor^^pietion time was BOF (65.31 days), conventional (66,27 days), and 
ino^Uiar (72,16 days). 

A one-way analysis of variance was. used to test these hypothesis by 
QofOParing completion times for modular and BOF (F(l,98) = 1,71) modular 

conventional (F( 1,102) = 1,62) and BOF and conventional (F(l,92) = 
.03). AG none of the F ratios were significant, all three experimental 
hypotheses were rejected. 



Summ^ 



lary 



1 



Of the eleven experimental hypotheses, only two were accepted as 

heing true. Specifically, modular students' performance was found to be 

« 

Significantly higher than BOF students and high ability reader^' per- 
formance significantly higher than that of low ability readers. Two 
c^tl^^r findings were significant but were opposite to what was expected, 
in "that xtonventional students' performance was found to be significantly 
higher than either the modular or the BOF students' performance. There 
^er^ no significant findings on the seven remaining hypotheses. 
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CHAPTER V 
THE FOOD SERVICE STUDY 
Introduction 

The Food Service study involves a comparison of texts in a behav- 
iorai objective format (BOF) versus innovative materials employing a 
high impact graphic (HIG) technique. T^jis chapter consists of six 
sections. First, two categories of Food Service students are described,, 
group assignments are specified, and student demographics by category 
and group of assignment are reported. This is followed by a description 
and statistical report of the measures used for hypothesis testing. 
Next, 'the method used in forming high, middle, and low reading ability 
subgroups is described. Then, each experimental hypothesis is described 
and tested, and an analysis of each is presented. This is followed by a 
brief summary of the hypotheses. 

Categories of Subjects and 
Assignment to Treatments 

Subjects were selected for the study and classified -in two cate- 
gories. 

The first consisted of students who had recently attended the Food 
Service 3-level resident school at Lowry Air Force Base in Colorado. 
Students who attended betweeji H August 19.76 and 16 December 1976 were 
j)Otential subjects for tliis study. 

4S 
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An instrument battery developed by the Extension Course Institute 
was administered to 210 of the Lowry resident student?? <^ iring the last 
five months of 1976. This instrument battery consisted of a course 
examination (CE) pretest for career development course 62250, a Cloze 
test for the 5-level Food Service Specialist bourse, a semantic differ- 
ential scale for measuring student attitudes toward the Food Service 
Specialist career field, and a biographical information sheet. 

The second category . consisted^ of students who were either directed 
dut^. assignees (those who attaln^^ -^heir ^-"^ ei "-^rrc.p J nee and 
had not attended the i ^sident schoo.l_> or student3 who had attended the 
resident school prior to August 1976. lor subjects in this category, no 
pretest data were available; consequently, they were randomly assigned 
tc one of the two treatment groups. 

Table 6 shows the source and assignment of subjects to the two 
treatment groups. A total of 131 usable sxibjects resiilted from the 215 
original assignments. As noted in chapter i, the expected attrition rate 
was 14 i-arcent, or approximately 30 subjects. The actual attrition was 
84 - about 34 percent. The higher than e:^pected attrition rate was 
probably duQ in part to the fact that parc.Lx: :.pation in the study was on 
a voluntary basis and in part to the recent civilian contracting of 
services normally performed by military food service personnel. 
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TAHLi: 6 



TREATMENT GROUP ASSIGNMENT AND SUBJECTS RETAINED 



Source 



BOF 



High Impact 
Graphic 



Total 



(Assignment to 
Treatment Groups) 

Category One (Resident Preparation) 
Category Two (Non-Resident Preparation) 
Totals 



53 
53 
105 



54 
55 
109 



107 
108 
215 



(Subjects Retained) 

Category One (Resident Pr eparat ion ) 

Category Two (Non-Resident Prep:n:iation ) 



42 
18 

60 



27 
71 



86 
131 




The 107 subjects from the first category were assigned to the BOF 
and high impact graphic treatment groups in a manner that assured as 
close a match as possible on Cloze scores and sex. The stratification 
of category one subjects on Cloze scores and sex and assignment to treat 
ment groups is shown in appendix L, The 108 students from category two 
were randomly assigned to eithebs,a EOF or high impact g^^aphic treatment 



In appendix I, demographics by category .and group assignment, 
including race, sex, educational level, marital status and number of 
dependents are shown. There it may be noted that with the exception 
of sex — seven females were assigned to the high impact graphic treat- 
ment group and none to the BOF — the distributions for category two 
are what would be expected of random assignments. 



Four measures were used to test the hypotheses. These were as 
follows : 



group. 



\ 



Measures Used for Testing Hypotheses 
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a. The BOF and high impact graphic course materials differed in 
objectives covered as well as the ways of presenting subject matter. The 
course examinations for each textual version contained 12U items, \00 

of which were common to the two examinations. Tnese items were used as 
a comparable measure of group performance. 

b. The semantic differential technique was used to obtain a mea- 
sure of each subject's attitude toward hi£ course materials. The ori- 
ginal attitude scale ranged from minus 3 to plus 3. 

c. A Cloze test was constructed from typical Food Service materials 
and administered to each subject. Cloze scores were used as a measure 

of reading comprehension. ^ 

d. A measure of completion time was obtained by subtracting the 
date on which each student received his course materials from the date 
on which the student completed the course examination. (A return post- 
card was included with all course materials with instructioi?! to indi- 
cate the date materials were received.) Statistics on these four 
measures ax^e shown in Table 7. 
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TABLE 7 

STATISTICS ON MEASURES USED IN THE STUDY 



Standard 



Score 


N 


Mean 


Median 


Mode 


Deviation 


R-^nge 


Reliability 


Per I orrnance 














* 


DUr 




/ J. • UD 


/ 0 


•77 


O . DH 


111 o o 




HIG 


71 


74,13 


75 


77 


6.99 


57-89 




Total 


131 


12.12 


74 


77 


7.91 


U7-89 


.80 


Semantic Differ 


2nt ia± 














bUr 


53 


O O *7 Ji 




31 


c It r\ 

b . hU 


111 1 1 n 




HIG 


69 


3 J . 00 


3G 


36 


8.06 


9-55 




Total 


122 


3U.57 


3U 


30 


7.53 


9-55 


NA 


Cloze 
















• Bor 


60 


20.07 


22 


2U 


7.39 


2-36 




HIG 


71 


21.28 


23 


2U 


7.16 /\J 


3-3U 




Total 


131 


21.02 


22 


2U 


7 . 2U ' 


2-36 


.79 


Completion Time 
















BOF 


29 


130.03 


127 




43. 74 


U8-232 




HIG 


40 


100.03 


lOU 




39.16 


17-225 




Total 


69 


112. 6U 


109 




1+3. U7 


17-232 


NA 



The similarity of each group is indicated by the statistics for 



each measure. The measures used showed no indication of violating the 
basic assumptions that underlie the statistical treatments used in this 
study (primarily the F-test as used in analysis of variance techniques). 

As in the previous case, the reliability coefficients for the 
perfoxnriance and Cloze scores were calculated by the Kuder - Richardson 
Formula 20, a standard measure of internal consistency of response. The 
performance score reliability (.80) had a standard error of 3.86, which 
would rosult in an individual score variation at the 99 percent confidence 
level of + 9.96. The Cloze score reliability (.79) had a standard error 
of 2.80, which would result in an individual score variation at the 99 
percent confidence level 6f _f 7.22. Reliability coefficients for the 
semantic differential instrument and completion times were not calculated. 
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The J'orTti ation of High» Middle > and Low 
^e^ding Ability Subgroups 

Two of the hypothesis required the formation of subgroups consist- 
ing. <'>^ rjtiidontG wh<'^ dem^J^stratod different levels of TH»ading ability. 
Thrc'o- lovpis ol r^eadin^r Nihility (high, middle, and low) were formed. 

A procedure paraXieiing that described in chapter IV was used in 
forming the subgroups. First, .it 'was noted (see Table 7) that the mean 
for the Cloze score was 2i.02. At the 99 percent confidence level, an 
individual Cloze scbre v'Ould vary no more than plus or minus 7.2. Con- 
,^idering this, high and iov/ reading ability groups were formed by using 
the Cloze score mean as ^ midpoint between the 'two groups, which were 
then separated by a 7 ppXnt spread. Thus, the high reading ability 
groups included all students who scored 25 or above on the Cloze test 
and the low reading ability groups were composed of thos& that scored 17 
or below. The mid^^le al?ility students consisted of students who scored 
within the range of 18 and 24. Frequency distributions of each group 
(high, middle, and low) within each treatment (BOF and high impact 
graphics) are shown in 'fables 8, 9, and 10. ; ♦ 

TABLE 8 * 



FREQUEKCy DISTRIBUTION OF CLOZE SCORES 
HIGH READING "ability 





BOF 


HIG 


EXACTLY MATCHED 


SCORE 


FREQ. 


FREQ. 


PAIRS 


3B 


1 






35 








3U 




1 




33 








32 


• 1 






31 


2 




2 


30 


U 


3 


3 


29 




1 




28 


3 


if 


3 


27 


2 


5 


2 


26 


5 


5 


5 


25 


3 


6 


3 



54 



U4 

TABLE 9 



FREQUENCY DISTRIBi;VION OF CLOZE SCCREF. 
MIDDLE READING ABILITY 





BOF 


HIG 


EXACTLY MATCHED 


SCORE 


FREQ. 


FREQ. 


PAIRS 


2U 


1 


6 


1 


23 


3 


1 


1 


22 


5 


5 


5 


21 




2 


2 


20 


1, 

H 


3 


3 


19 


1 


3 


1 


18 


3 


2 


2 






TABLE 10 






t REQUENCY 


DISTRIBUTION OF 


CLOZE SCORES 


• 




LOW READING ABILITY 




BOF 


HIG 


EXACTLY MATCHED 


SCORE 


FREQ. 


FREQ. 


PAIRS 


17 


1 


1 


1 


15 


3 


U 


3 


15 


3 


3 


3 


1h 


1 








1 

X 


1 


1 


12 




.1 




11 


2 


2 


2 


10 


1 


3 


1 


9 


2 


2 


2 


8 


1 






7 




1 




6 


2 






5 












1 








1 




2 


1 







Hypotheses Tested 
Five hypotheses were formulated and tested for the Food Services 
portion of the study. A description of each hypothesis and resulting 
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: in-.: irir.^"> ir, given in this flection. Tho procedurt^ : o.l ] .-^w- tor iMrli w.).-. 
to stare the- hypothesis, present the statistical techniquer^ •'];-ed :'oi 
testing the hypothesis, and cite the findings. Analys^^ of variance 
techniques were based on Ferguson's description of one and two way 
classification."^ ^^o^^ each experimencal hypothesis tested, significance 
at or beyond the .05 confidence level was used for acceptance. 

The first hypotnesis pertained to tx^eatment effects on student's 
performance. 

Effects of instruction on perforniance 

Hypothesis one was that students who study high impact graphic course 
materials would show significantly better performance scores than those 
who study BOF materials, 

A one-way classification, analysis of variance, was used to test 
the hypo^hN3sis. A comparison was made between performance scores for 
high impact graphic students (mean = 74.13) and BOF students (mean = 
71.05). The comparison indicated a significant difference - (F(l529) = 
5^57, P < ,05) favoring the high impact graphic students' performance. 
Thus, the first experimental hypothesis was accepted as being true. 

Effects of reading ability on performance 

The^ second hypothesis was that high ability readers would perform 
significantly better than low ability readers. To test this, students 
were grouped into high and low reading ability groups depending on their 

George A. Ferguson, Statistical Analysis in Psychology and Educa- 
tion (New York: McGraw-Hill Book Co., 1971), pp. 208-45. 
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.■lo::v' tr-r, * vror^r. . r'tud*:*nTs who scored above 24 were includOv": in the 
h'^h re.^diri£ ahility r^roup, and those that scored below IS in the low 
r-"^ad Lng ability group. The frequency distribution of the Cloze scores 
for the high and low reading ability groups is shown in Tables B and 10. 
A one-We-jy analysis of variance was used ro test this hypothesis. A 
comi^'ar I r. on wai. made between the performance scores for high reading abi- 
J i ty students (mean - lb. 12) and low reading ability students (mean - 
68. 61). The comparison indicated a highly significant difference 
(r(l,86) - 22.27, p < .01) favoring the high ability readers' performance. 
Thus, the second experimental hypothesis was accepted as being true. 

Unique effects of treatments and reading ability levels 

The third hypothesis was that high ability readers would show 
little difference in performance from one treatment to another while 
middle and low ability readers would demonstrate best performance using 
high impact graphic course materials and poorest performance using BOF 
materials. To test this, students in each treatment group were divided 
into three subgroups depending on their Cloze scores. Within each 
subgroup and bctwec ^^ach treatment group, the Cloze test scores were 
used to form matched pairs of students. When exactly matched pairs of 
students* Cloze scores did not exist, the closest possible match was 
sought. The high reading ability group consisted of best matched 
students who scored 25 or better on the Cloze test, the middle reading 
ability group consisted of best matched students who scored within the 
range of 18 to ?U, and the low reading ability group was made up of best 
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A twr^-wa-/ :n.ilv^>Ir. oi varianre W3" U!;ov] to r*':,t iot* i '~ifnit'ic.H:t 
!nt«'r-t«:t ion '^Itoct (rreatncnt by reading ability level). A conipari:;on 
wa:; made of the variability of the difference between the subgroups' 
performance score means at each level across treatments. This compari- 
son is depicted in Figure U, where the relative standing of performance 
ncore means for subgroups is shown. The variability of differences 
was not significant {F(2,11'0 = 1.U2) and consequently, the third experi- 
mental hypothesis was not accepted as being true. 

FIGURE U 
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Effects of format on student attitude 

The fourth hypothesis was that students who studied the high impact 
p,raphic course would display a significantly more positive attitude to- 
waixl their training materials than those who received the BOF course. 

A one-way class if icatior. analysis of variance was used to test for 
a significant difference on the semantic differential instrument scores 
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fcetween high impact graphic (mean score - 35.99) and BOf (mean score - 
32.74) students. Figure 5 shows the relative standing of high impact 
;^,raphic and BOF students on the original semantic differential scale 
ranging from plus three to minus three. The comparison indicated a 
significant difference (F(i,120) = 5.81, P < .05) in attitudes with - 
the high impact graphics students being the most positive. Thus, the 
fo\irth hypothesis was accepted as being true. 

FIGURE 5 

DISLIKE -3 -2 -1 0 +1 +2 +3 LIKE 
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Effects of format on completion times 

The fifth hypothesis wa:> tnat the completion time in days for stu- 
dents using high impact ^^raphic mate^^ials would be significantly shorter 
than for students using BOF materials^ 

A one-way analysis of variance was used to test for a significant 
difference in completion time between the high impact graphic (mean = 
100 days) and BOF (mean = 130 days) students. The comparison indicated 
a highly significant difference (F(l,t~7) = 8.95, P < .01) in comple- 
tion times, with the high impact graphics students requiring the least 
time. Thus, the fifth hypothesis was accepted as being true. 

Summary 

In summary, four, of the five experimental hypotheses were accepted. 
Student performance was found to be significantly better on the high 

5£' 
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irij'.u f rinir,.*' ;n, 1 1 < iri j : { luii: Hm' riiat ia 1 . . t^m^I-m':; 

wc'r<' round to perform significantly bett-^r on end of course exams than 
less able readers, students' attitudes toward the high impact graphic 
material were significantly more positive than those toward the BOF 
material, and the number of days required for completion were signi- 
tic^ntly fewer for the high impact graphic students. The hypothesis 
about the treatment by reading ability level was not supported. As 
expected^ able readers showed little difference in performance on either 
the high impact graphics or BOF course and less able readers showed 
hotter performance on the high impact graphics course, however, the 
interaction effect of these differences in performance was not signifi- 
cant. 

In the next chapter, support of 'the findings for the sixteen 
hypotheses (eleven for the Materiel Facility specialists and five 
for the Food Service students) will be sought through a review of 
student surveys. 
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CHAPTER VI 



STUDE.NT OPINION SURVEYS 
Introduction 

Upon completion of their respective course examinations, students 
who participated in the Materiel Facilities and Food Service studies 
were asked to respond to a survey instrument consisting of twenty-five 
items. In this chapter the interpretation of the survey responses for 
each study will be given. First, a general overview highlighting the 
innovative courses will be presented. Then a more specific perspective 
compares survey results to relevant findings resulting from the hypotheses 
tested in each study. 

For the interested reader, a description of how the survey instru- 
ment was interpreted is given in appendixes M, N, and 0. Appendix M 
describes how the survey items are classified into types and subtypes; N 
reports h6w the student responses to the survey instrioment are regarded 
and lists each survey item and its pertinent response data, and 0 des- 
cribes and lists the frequency of occurrance of arrays and treatment 
rankings for seven item clusters. Appendix P graphically depicts the 
percentage of students in each Materiel Facilities group who responded 

4- 

negatively to each survey item. Item numbers are sequenced to show 
progressive increases in negativity by students in the innovative group 
(modular BOF). Appendix Q reflects similar information for the Food 
Service survey. 



An Overview of Innovative Courses 

The materiel facilities and food service studies each had treatment 

groups in which innovative textual materials were used. For the Materie 

Facilities study this was the modular group and for the Food Service 

study the high impact graphic group. 

From a global perspective, students from the innovative group for 

■ I 

each study gave more positive responses to the survey items than their 
corres{)onding BOF counterparts. An interpretation of the student res- 
ponses resulted in the following contentions* The Materiel Facilities 
conventional course war: more positively regarded than the modular course 
and th»r^ modular course was more positively regarded than the BOF course. 
F'or the Food Service study the high impact graphic course was found to 
be more positively regarded than the BOF. Thus, in each study, the 
innovative courses were found to be more positively regarded than those 
in the behavioral objective format. 

The Materiel Facilities Survey 
In this section, analysis will be limited to the Materiel Facili- 
ties study §nd the student responses to the survey items will be viewed 
from a more specific prospect. Six clusters of items were formed on the 
basis of objective similarity and are interpreted with respect to the 
relevant findings that resulted from the eleven hypotheses (see appendix 
R) tested in chapter IV. 

Difficulty of Texts 

» 

Tlie first cluster contains six items (1, 5, 6, 7, 8, and 9) that 
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relate to the difficulty of texts. The following interpretations re- 
salted frori student responses to these items. 

More students from the BOF course than from either of the others 
indicated the text matei^ials were difficult considering their present 
level of training. They also found the BOF version hard to read and 
difficult to understand. In the opinion of the students, the BOF 
version contained coo many technical terms, had an excessive number of 
abbreviations, and frequently employed technical terms that were un- 
defined . 

More modular text students found their materials too difficult for 
their present level of training than did conventional students; this 
seemed related to modular^ student perceptions of excessive technical 
terms and abbreviations in their texts. 

More conventional than modular students indicated their materials 
were difficult to understand. This probably relates to conventional 
student perceptions that their texts are difficult to read. 

All subjects shared the view that their materials frequently 
employed technical terms or acronyms that were undefined; this was 
expressed by over 60 percent of the students from each group with the 
difference between groups leos than two percent. 

The above interpretations of the items had a direct bearir^ on 
hypotheses 1, 2, and 3. The perceptions of least difficult materials 
for conventional, less for modular and greatest for BOF were consistent 
with an acceptance of the first hypothesis and the rejection of the 
second and third. 
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Textual coverage 

The second cluster contains four 5 loitis that I'-eldte tc textual 
coverage. These are items 2, 3, and 10. The following- conclusions 
were derived from student responses to these items. 

30r students believed, more so that modular students, that the 
course materials were deficient in illustrations and foldouts, and that 
behavioral objectives were inadequately covered in the text and poorly 
tested. BOF and modular course materials both depend heavily on be- 
havioral objectives as study guides, while conventional objectives were 
not rigidly organized and thus could not be meaningfully compared here. 
This interpretation was consistent with the first hypothecs in that a 
more adequate coverage of course objectives and better illustrations and 
foldouts describe conditions that most likely would result in an an- 
crease in student perf om.ance. 

Relevance of Texts to Job 

BOF and modular students expressed less satisfaction than conven- 
tional students on several issues: (1) they believed that their courses 
contained more materialXthan needed at their particular stages in their 
Air Force careers, (2) they believed that their courses contained more 
material than needed to do their jobs (over 50% of the topics discussed 
were said to not relate to their present jobs), and (3) they considered 
their courses to be unrealistically complex. These students also felt 
that few of the different types of equipment discussed were used in 
their jobs, and that few of the tasks described were ever performed. 

In short, a larger percentage of students indicated that conven-- 
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tional course materials were more closely related to the actual jc:. than 
•ii St, .dents ir. nodular or BOF -."ersions. 

In contrasting inodular and BOF students, it was learr.ed that r.cr^- 
r.odular than BOF students regarded the scope of their r.aterial as too 
broad. Modular students also felt that their jobs required less expc- 
:-.ure to the equipment discussed in their texts. 

In shoi^r, BOF students, more so than modular students, agreed that 
most of the topics discussed in the course did not relate to the job, 
that the tasks described were more complex than those on the job, and 
that many of the tasks were rarely perfcrmed. 

The above speculations support hypothesis one in that the lower 
performance found for the HOP students is compatible with a larger 
percentage from that group than from the modular group- perceiving their 
course to be overly complex and unrelated to the job performed. 

D ifficulty of coui'se examinations 

-he fourth cluster of items related to the difficulty of course 
examinations and consisted of items 21 and 22. What follows is an 
interpretation of student responses. 

Many modular students, fewer BOF students, and still fewer conven- 
tional students indicated that they had difficulty understanding the 
course examination questions. Most students regarded their test as 
being somewhat difficult. It was noted that less than 7 percent of any 
group indicated that the course examination was too easy to really test 
one's knowledge of the course materials. 
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ccn r-?r.* icn that, fewer conventional students pprceivod tIip 
•■ --xar. : nar lor. as reing difficult than did Giudontn in eithei' ^ r h^^ 
;tner croup--. , iz consistent with the f:econd and third hypotheses. It I', 
inconGi'-jtent with the first in that the rank order on exam perf ori?.ance 
was conventional highest; modular next highest; and EOF, lowest. 

'curse examination coverage^ 

The fifth cluster of items pertained to the course examination 
coverage and consisted of items 19, 23, and 25. A greater percentage of 
BOF ^han modular students and still fewer conventional students indicated 
that the course examination asked questions about material not contained 
in the texts. This interpretation is consistent with the findings for 
hypotheses 1, 2, and 3 in that the performance score rank orders agree 
with the responses. Tne next two observations were of particular 
iniportance. First, more than half of the modular and BOF students, but 
only a fifth of the conventional students indicated that most course 
examination questions did not come directly from the volume review 
exercises. Second, a greater percentage of students from the conven- 
tional course, fewer from the modular course, and fewest from the BOF 
indicated that they prepared for the course examination by studying only 
the volume review exercises or only the text. As a result of these two 
observations, an analysis of the relationship of volume review exercises 
and course examinations was made for each course version. 

It was found that a U4 percent repeti'cion of identical items 
existed between the conventional course examination and its correspond- 




Ir.i.-. vclur.e review exercise. There was no such overlap in eithf.r the HOT 
:r *h^- r.cdul-ar exar. inat ions and ^heir corresponding volurre review exer- 
-:i-:.e-.. Th^'^^e findings Lear heavily on the second and third hypotheses 
-and could very v/<?ll account for the unexpected reversed findings for 
these two hypotheses. 

Course examination relevance to the job 

The sixth cluster of items pertained to course e>i^mination rele- 
vancy to the student's job. The cluster consisted of items 20 and 2U. 
It was noted that a greater percentage of BOF and modular students than 
conventional students indicated that the coui^se examination neither 
covere(5 the most important concepts relating to their jobs, nor tested 
subjects important to their jobs. The above interpretation would most 
Likely have bearing on the rejection of the ^^eventh and eighth hypotlieses 
as the student's attitude about the course materials would be affecced 
by their perceptions of the adequacy of course exajninat ions . 

The Food Service Survey 
This section analyses the survey for the food service study. 
Again, as in the proceeding section, student responses to survey items 
will be viewed from a specific prospect. Six clusters of items are 
interpreted and compared to relevant findings that resulted from hypo- 
theses (see appendix R) tested in cha\>ter V. 

Difficulty of texts 

The fi^st cluster contains six items\that relate to text difficulty 
(items 1, 6, 6, 7, 8, and 9). The following speculations resulted from 



:r. ; r.* '.t:, r -'rtar ion of the student's responses to these iteris. 

■■■::.T^ra 1 1 V cpeaking, the EOF ccurs*^ was consiaered ris l-eir:^ T.cr^ 
: ' "^ jlt than *r.*^ high i-pact j?;raphic version. Mere 5?tu kj^nts frorr: the 
!-':r c .'ur ::e indicated the text r.aterials were diffioalt consi.dering 
their present level of training. They also found the BGF difficult to 

understand and hard to read. In the opinion of the students, the BOF 
contained too many technical terms and had an excessive number of abbre- 
viationr,. However-, technical rerms were considered as being more 
frequ<"'ntly undefined in the high impact graphic course. J/i spite of 
this, this version was considered as being generally less difficult. 
TTiis interpretation Is consistent with the findings for hypotheses 1, 3, 

i'cr hypothose:; 1, and S, the findings of significantly better 
;.or romance, r.ore positive attitude, and quicker completion time by high 
impact paraph ic students >as consistenr with the survey interpretation 
that thir. coui'se was generally considered to be less, difficult. For 
hypother^is 3, the higher (but not signif .rcant ) performance by less able 
readers on the high impact graphic 'version was consistent with the 
survey interpretation that this course was easier to read. 

Textual coverage 

The second cluster ^contains four items that relate to textual 
coverage; items ?, 3, U, and 10. The following conjecturs were drawii 
from an interpretation of the student responses to these items. o 

With respect to the coverage of stated objectives, the high impact 
graphic course was generally considered to be more adequate than the 
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BOr. More high impact graphic than BOF. students had the opinion that 
the questions at the end of each objective actually tested the objec- 
tive. Also, EOF students believed that there had been inadequate use oJ 
illustrations and foldouts. 

To briefly summarize, the BOF seemed to be covering the right 
things but was lacking in the manner of presentation while the high 
impac; -^ -ohic course had a better presentation but was lacking in 
C'.v j'vai^e. This interpretation would most likely have bearing on the 
first and fourth hypotheses as a course that is better presented would 
result in "-ligher performance and be better received. 

Relevance of texts to job 

The third cluster of items related to the relevance of textual 
materials to the actual jobs performed by students. This cluster con- 
sisted of items 11, 12, 13, 14, 15, and 16, 

BOF students believed that their course contained more material 
than needed at their particular stage in the Air Force and more materj.d-. 
than needed to do their jobs (over 50 percent of the topics discussed 
were believed to be nonrelated to their jobs). On the other hand, high 
impact graphic students felt that few of the different types of equip- 
ment discussed were used on their jobs, and that few, of the tasks de- 
scribed were ever performed. To briefly summarize, the BOF course 
seemed to have too much coverage with respect to the job while the high 
impact graphic course had too little. This interpretation would most 
likely have some bearing on the fifth hypothesis as completion time is 
related to the amount of material covered. 




Difficulty of course examinations 

The fourth cluster consisted of items 21 and 22 and related to the 
difficulty of the course examinations. 

With respect to examination difficulty, BOF students considered 
their examination as being too easy to test their knowledge of the 
material while the high impact graphic group had more difficulty under- 
standing the questions. It is noted that the percentage of students 
from the respective courses voicing this opinion was relatively small 
(less than ?1 percent for any course). Consequently, the interpretation 
of these items have little, if any, bearing on the hypotheses. 

Course examination coverage 

The fifth cluster of items pertained to the course examination 
coverage and consisted of items 19, 23, and 25, 

BOF students believed that the course examination asked questions 
abou' materials not covered in the text. When prepciring for the course 
examination, BOF students also believed examination questions came 
directly from the volume review exercises and consistent with this 
belief they undertook a less comprehensive review uf course materials 
prior to taking their exams. The first interpretation that exam ques^ 
tions were not covered in the text would be supportive of a positive 
attitude toward the course materials and consistent with the fourth 
hypothesis. However, the belief that examination questions came di- 
rectly from the volume reviews would most likely have the opposite 
effect. 
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Course examination relevance to job 



The sixth cluster of items pertained to the course examination 
relevancy to the student's job. The cluster consisted of items 20 
and 

Although the high impact graphic material was considered less 
related to the job than the BOF, the high impact graphic course examina- 
tion was considered to be better related to the job than that of BOF. 
High impact graphic students believed that the course examinations ' 
covei-^ed the most importaht concepts relating to their job and that most 
examination questions " were about topics important to their job. This 
interpretation would most likely have a bearing on the fourth hypothesis 
as acceptance of the course materials is related to student feelings 
about the appropriateness of the course examination. 



A review and interpretation of the student survey was made for 
students in the Materiel Facilities and Food Service courses. First, 
an overview of the survey items was presented. In general, it was found 
that innovative courses were more favorably regarded than BOF courses. 
Then a description of the item interpretations as they related to the 
hypotheses tisted in chapters U and 5 was gi.ven. In most cases, it was 
found that the interpretations of survey items were consistent with and 
supportive of one or more of the h3^otheses. 



Summary 
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CHAPTER VII 
SUMMARY AND CONCLUSIONS 
Introduction 

The preceding chapters have presented an analysis of innovative 
correspondence mater ia3.s in two Air Force career fields. In the appren- 
tice Materiel Facilities Specialist's course, a three-way comparison was 
made between conventional texts, materials in the BOF, and an innovative 
modular BOF text. In the Food Service Specialist's course, a comparison 
was made between texts in the BOF and those using the innovative high 
impact graphic technique. This chapter summarizes the findings of. each 
of these studies and states the conclusions drawn. It is important to 
recognize that these conclusions are based only on the two career 
development courses studied; therefore, generalizations beyond wh^t has 
been experimentally observed should be made with caution. The findings 
suggest, however, that the innovative materials studied have varying 
degrees of value in terms of student achievement and attitude with 
potential for further application in other Air Force 'career specialties. 

Restatement of Hypotheses 
The experimental hypotheses as stated in chapter I were as follows: 
a. That students who receive instruction by way of modified 
CDCs (newly developed) will perform significantly better on course 



examinations than those who receive instruction by conventional texts. 

That high ability readers will perforin significantly 
better than low ability readers on each respective field test course 
examination. 

c. That, on the Food Services course examinations, high abi- 
lity readers will show little difference in performance depending on 
training media while low ability readers will perform significantly 
better using the contemporary narrative/enhanced graphics version, 

d. That, on the Materiel Facilities course examinations, high 
ability readers will show little difference in performance depending on 
training media whereas low ability readers will demonstrate poorest 
performance using conventional texts, intermediate performance with BOF, 
and highest performance using modular BOF texts. 

e. That students receiving modified text instruction will 
display a significantly more positive attitude toward their training 
than those who receive instruction by conventional CDCs. 

f . That the innovative texts will be completed in a shorter 
average time than current conventional and BOF volumes. 

Each hypothesis was tested using analysis of varian^.e techniques 
with a .05 significance level (chance occurrence no greater than 5 times 
in 100) as the criterion for acceptance. 

Materiel Facilities Course Study 
In the Materiel Facilities course (CDC 6H531), a three-way compari- 
son of conventional texts, the BOF, and the innovative Modular BOF was 
conducted. The Modular BOF involves chapters that may be studied in any 
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order (mo-iuleG), liberal illustrations, and frequent summarizing notes. 
A minimum of 100 students were enrolled in each group with usably data 
yielded as follows: conventional, 75; BOF, 75; and modular BOF, 83, 

Student subscores were then compared by test format with these mean 
results: conventional, 74.31; BOF 69.91; modular, 69.95. The mean 
score of each group using behavioral obj-«ctives was significantly lower 
than that of the conventional group • On the basis of that finding, 
hypothesis "a" was rejected. While unexpected, this is not difficult to 
explain. The advantage of behavioral objectives lies in their ability 
to focus curriculum planning and to communicate instructional intent to 
the student. Research has not consistently shown that behayioral objec- 
tives result in greater student achievemer.C. Further, the opinion 
survey conducted as a part of this study suggested that students in ^:he 
BOF and modular courses felt inadequately prepared for their course 
examinations py each volume review ex^ercise (VRE). Later investigation 
showed a 44 percent repetition of VRE and course examination items in 
the conventional course, but no such overlap in either the BOF or innova- 
tive modular version. This test overlap may have biased findings 
associated with the conventional materials. 

The average subscores of low reading ability students was 68.08, 
and that of the high ability group was 73.75. The 5.67 percentage point 
difference in subscores was significant, and hypothesis "b" was accepted. 
Student readi xg ability was again shown to be a student characteristic 
significantly affecting test performance. 

Hypothesis "c'* did not apply to the Materiel Facilities course. 

When students were stratified according to reading ability, both 
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high and low groups perforined best using conventional texts, poorest 
with the BOF, and intermediately using modular BOF mat-erials. There was 
no significant interaction effect between any group, therefore, hypothe- 
sis "d" was rejected. While the innovative modular group did not out 
per.'orm the conventional group as'hypothesized, its performance over the 
BOF group was significant and should be noted. 

Differences in student acceptance of the three textual versions 
were not significant, therefore, hypothesis"e" was rejected. 

The average number of days required for course completion was 66 
■for the conventional version, 75 for the BOF, and 72 for the modular 
format. These differences were not sisnificant, and hjrpothesis "f" was 
rejected. 

Food Service Course Study 
In the Food Service career field (CDC 62250), a comparison was made 
between students using a text in the Behavioral Objective Format (BOF) 
and students using the newly developed high impact graphic text. Of 106 
students enrolled in the BOF, 60 completed in time to^ield usable data; 
of 109 in the high impact graphic version, 71 produced usable data. 
Performance was measured by subscort on the examination items common to 
both versions. Student recept iveness to texts was obtained using the 
semantic differential technique (polar adjectives, such as "deep \ — 
shallow"). Reading >:omprehension was measured by scores on a fill-in- 
the blank examination, a procedure called the Cloze technique. Each hy- 
pothesis was then tested separately. 

: J 
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^ students were again divided into high and low reading ability 
grovps and common item suhscores were compared. The low reading ability 
group averaged 68,61 correct; the high reading ability group lb. 12. The 
7,11 percentage point difference was statistically significant, there- 
fore, the hypothesis "a" was accepted. 

Students were then compared by textual treatment groups while dis- 
regarding reading ability, Subscores for the BOF students averaged 
71.05, compared with 74,13 for the high impact graphic students. The 
3,08 percentage point performance difference by students using the high 
impact graphic text was significant, therefore, hypothesis "b" was 
accepted. 

Next, students were compared using different textual formats while 
considering reading abilities in a treatment by level* design. As hypothe- 
sized, able readers perforined well using either text while low ability 
readers showed some performance increase using the high impact graphic 
materials. However, this increased performance was not significant and 
caused rejection of hypothesis "c". 

Hypotheses "d" did not apply to the food service course. 

General student acceptance of this innovative high impact graphic 
material was significantly greater than that of the BOF based on, semantic 
differential test scores. Hypothesis "e" was therefore accepted. 

Course completion tim*^ was: defined as the time between coiirse 
package receipt by the student and the date of course examination. The 
BOF students required an average of 130 days to complete, whereas the 
high impact graphics students averaged 100 days. This difference is 
significant, and hypothesis "f* was accepted. 
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Conclusions and Recommendations 

bcveral observations and conclusions can be drawn from this field 
test. First, the procedures used to gcither data worked, and could be 
used again to evaluate other innovations as the need arises. Second, 
when the high impact graphic text was used by students of limited read- 
ing ability, it worked as anticipated and worked well. This format 
could be usefully applied to other career r^avelopment courses in which 
reading problems are known to exist. Finally, though the modular 
approach did not demonstrate all anticipated performance advantages, it 
did show slight improvement over the present BOF. The difference is 
enough to justify continued development to expand use of the modular 
concept in selected areas. 

The following recommendations can be derived from this study: 
First, that the Extension Course Institute apply the field study methods 
developed here to the validation of other innovations in its corres^ 
pondence program. Second, that Air Training Command preparing age- cies 
be encouraged in the extensive uso of illustrations in Career Dev?l vp- 
nent Courses, particularly when populations of low-ability readers are 
believed to exist. Finally, that self-contained study modules be 
developed wjienever this is compatible with the subject matter, to facili 
tate direct text support of on-the-job training. 
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CHAPTER 3 

Appendix A 



Conventional Text 
(Materiel Facilities Course) 



Supply Catabgirig Publlcationt 



RECORDS FOR a recent calendar ye&r show 
that the Air Force had approximately two 
million active cataloged items in its inventory. 
Th se itemo, il is estimated, were involved in 
the pro»:es8ing of million bi pply 

transactions between depots and bases. 
Furthermore, this figure does not include the 
innumerable tran'^actions that were made 
between the hundreds of base level accounts 
t'nd their thousands of local customers. Every 
ove of these transactions requires accurate 
comimunication of spet^ific supply data from 
one. activity to another. Unless there is a 
common language between the user on ono 
hand and the supplier on the other, it is 
almost impossible for an effective transac'don 
tQ occur at any level. To meet this need for a 
common language relating to items of svpply, 
the Department of Defens«^ and tiu ^^xt F^rce 
prepare, pubBish, and distribute the various 
cataloging piibilications included in the USAF 
supply catalogiing system. 



2. The ms^or arm of ail supply 
management functior^s is to provide each 
indhridual or organization with the items 
needed when they ve waifited, and in ^he 
condition, quantity, and location required. 
The different kinds of cataloging publications 
used by the Air Force provide supply 
personnel and their customers with 
information and data to insure positive 
identific ition and efficient management of 
supply items. Supply catalogs contain orderly 
UstiRgc^ of the millions of active Air Force 
items. Other kinds ol publications in this 
gystem contair inftJRSation and ?P.structions 
which help yot; to locate and effectively use 
the item idenfil'cation and management data 
Included in the stopk list and supply catalog 
listings. You will fin<J that the knowledge you 
gain from your study^ of supply cataloging 
publications will be ^used daily in the 
performance of your duties. 



1. Federa Supply Item Qassification, 
Identification, and Cataloging 

1-1. In order to use supply catalogs, you 
need to first understand Uie basic piinciples 
of property classification and item 
identification. Materiel and supplies in 
tremendous quantities are required by the 
military forces of the United States both in 
peacetime and war. Billions of dollan, 
thousan^ls of pieces of equipment, and ' 
millioiis of parts and other ty pes of supplies 
must be managed. For many yecrs, each of 
our military departments used its own system' 
of management to classify, identify, describe, 
and catalog items of supply. Now, under the 
Federal (Defense) Cataloging Program, thi 
military forces and other Government 
agencies use the same system. 

> 1-2. Federal SappJ^ Classification. Whe^ 
you consider the vFst number of items in 
Department of Defense (DOD) supply 
dystJUiS and the worldwide scope of 
operations, it is evident that supply managers 
c:mnct manage their systems on an 
itam-by-item basis. To provide a manageable 
system, items must be divided into "fwnily" 
groups. Management of our coi^lex BOD 
mpply inventories, tlrien, begins with the 
cbissification of all items. 

1-3^ The Federal supply classification 
iFSC) system provides an orgaiiiz d structure 
iot grouping and relating all tue different 
• ids of items on a "commodity" or 
"class-of-item" basis. The FSC system is 
designed to serve the functions of supply and 
is sufficiently comprehensive in scope to 
permit the classification of all items of 
personal property. (Personal property is 
briefly defined as aU conmi6dities other than 
real estate.) This FSC structure, which 
includes approximately 76 groups, subdivided 
into about 585 classes, could be expande ' to 
99 groups with 99 classes within each group if 
necessary. 
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1-4. The principle of commodity 
classification provides for the grouping of 
items undar na-nes 6r titles according to their 
use or purpose. FSC groups, just as the 
separate departments in a large depar^«nent 
store, contain such m^or groupings of 
commodities as clothing, tires and tubes, 
photographic equipment, furniture, ^tz. 
Commodity classification further provides fcT 
the breakdown of commodity groups into 
classes ai necessary. Each class consists of 
commodities which are relatively similar in 
one way or another. For e: ample, the group 
(FSG 5100) entitled "Hanc Tools" is broken 
down into classes entitled "Hand Tools, 
Power Driven" (FSC 5130), etc. The items 
within a class may be related in respect to 
their physical or performance characteristics. 
In some classes, the relationship of items moy 
be that they are requisitioned or Msuad 
together or that the/ are grouped together for 
supply management purposes. 

1-5. In addition to name or title, 
commodity classifications are identified by 
assigned numerical codes. A 4*digit coding 
structure is used for chis purpose. The first 
two digits of the FSC code number identify 
the gi'oup, and the last two digits i6m^ty the 
classes within the group. For example, the 
4-digit codes assigned to classes in Group 15 
begin with "15" (1510, 1520, 1530, e^c), 
while those assigned to Group 53 begin with 
"53" (5305, 5306, 5307, etc.). Note 
that the last two digits of Lhe FSC code 
identify a specific class within gto'jp, as 
indicated for the Tollowing class breakdowns 
in Group 53, Hardware and Abrasives: 

Class 530tr -Screws 

Class 5306— liolts 

Class 5307— Studs 

Clar? 5310— Nuts and Washers 

1-6. Code numh«rs assigned to Federal 
supply groups begin ^vith 10 and continue 
consecutively through 99, except for certain 
unassigned nun\bers, as shown in the. 
following partial list of FSC groups: 

Group 

85 ToUetric* 

86 Un.vsigned 

87 Agriculture Supplies 

88 Live Animals 

89 Suteistence 

90 Unassigned 

91 Fuels, Lubricants^ Oils, and Waxes 

92 Unassigned 

93 Nonmetallic Fabricated Materials 

Unassigned numbers, such as 86, 90, and 92, 
can t-c used to identify any additional 



commodity groups that might be est*»blished 
because of technological advances or oth^ 
desired changes. 

1-7. Code numbers are so assigned to 
classes within a group as to make it possible 
to expand the number cf classes as this 
becomej necessary. In most groups, gaps have 
been left bet ween some adjacent class code 
numbers so that new classes may be inserted 
later in logical sequence. To illustrate, let us 
consider Group 34, Metaiworking Machinery. 
The classes in this group begin with 3405; 
skip to 3410; continue with 3411, 3412, 
3413, 3415, 3-il6, 3417, 3418, and 3419; 
Dkip to 3422, tiien to 3424; and so on. Gaps, 
such as 3420 pnd 3421> can be used for later 
expansion if necessary. 

1-8. Item Identification. In cataloging and 
supply operations, identification of items 
n^ust be positive and accurate so that the 
identification of one item C!i*mot be confused 
with ny other similar or like item. Federal 
item identification consists of the minimum 
data necessary to ^tablish the essential 
characteristics of an item and to differentiate 
it from every other Item of supply. Item 
identification includeb/four principal elements 
of data: (1) item naifae, (2) descriptive data, 

(3) Federal item idei\tification number, and 

(4) Federal stock nun?.ber. 

1-9, Item name. TL<; a-m name is the 
naiae approved und«r the Federal Cataloging 
Program for official use thrc a^out the entire 
milita*/ defense systems. After necessary 
research, a single item name is selected for 
each article so that ail DOD activities cai\ 
speak the same language. In other words, they 
use the same name for the same article. The 
item name answors the question, "What is it?" 
Depending on the nature ox kind of iten^, the 
item name may consist of a basic name or a 
basic name with modifiers. 

1-10. The basic name may be a noun word, 
such PESISTOR; or it may be a noun 
-^rase, ouch as CLEANING COMPOUND, 
PANEL \SSEMBLY, etc, Mo.lifien are 
necessary where different kinds of items have 
the samt: basic name such as CAMERA, 
STILL PICTURE; CAMERA, MOTION 
PICTURE; CAMERA, TELSVISONi etc. 

1-11. Descriptive ^data. WhUe the item 
name tells what the item is, additional, 
descriptive data is usually necessary to 
identify each item from every other item of 
supply. The complete description required to 
identify an item, then, consists of the item 
name, followed by descriptive information. 
For example, several items may have the same 
item name but may diffef in size, in color, or 
in some other v/ay. 
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Behavorial Objective Format 
(Food Services Course) 



Cooking Methods 



TO COOK IS to expose foods to the riction of heat 
in order to make them n.ore digestible. There are 
various ways of applying heat to foods. The method 
used depends a great deal upon whether the item ia 
lough or lender. If the item is tough, a lengthy cook- 
ing period is required to make the it^hi hiore 
appetizing and digestible. If the item is tender, \ 
faster cooking measure is used. 

2-1. C okhig Methods 

Cooking is performed by two basic methods- 
dry heal and moist heat. One of these is used 
regardless of the product v^ookcd, be it meat, 
vegetable, or pastry. Wc will discuss these two cook- 
ing methods briefly in tije following paragraphs, 
and, later on, go intojmore detail when we cover the 
preparation of individual foods. 

203. Indicate whether sUtements pertaining to dry- 
heat cooking methods are true or false. . 

Dry-Heat Cooking. When you cook with dry 
hept, no liquid is addfcd. In most cases, the product 
itself suppli*^ enough fat and/ or juices to keep it 
from burning. Dry heat in used when cooking tender 
cuts of mefai, fish, youhft poultry, some types of 
vegetables, and n.ost pastry and bread products. 
Dry-hcat cooking procedures are explained in the 
folioving^aragraphs. 

' Roasting, one of the more popular cockmg 
procedures, is done in an oven. The food being 
roasted must be kept uncovered, otl;crwisc the food 
will steam. The term "roasting," although essential- 
ly the same as baking, generally applies to meat 
items, whereas "baking" applies to fish and dcugh 
products. 

Broiling is cooking by direct heat over coals or 
under gas flame 6r an electric heating unit. This 
••'^oking procedure is used mainly ih the preparation 
of meat, poultry, and fish. It is a very quick artd sim- 
ple method, but it is linaitcd to the cooking of verj' 
tender meat cuts, such as steaks and chops. 

Pan or griddic broiling is cooking on top of the 
range in a fiying pan or on a griddle, using for grease 
only the fat of -the meat itself. 

Pan frying is done with just enough added fat 
(grease) to cook the food. Only certain foods arc 



adapted to pan frying. Do not use this cooking 
method unless it is prescribed by the recipe. 

Deep-fat frying is cooking food by completely 
covering it with hot fat. In deep-fat frying, foods are 
cooked quickly enough to prevent their absorbing 
any of the cooking fat. Holding the cooking fat at 
the correct temperature is very important when us- 
ing this method. Too high a temperature causes the 
product to become hard and dry; too low a 
temperature allov s the product to absorb the cook- 
ing fat. 

Recommended dry-hcat cooking temperatures 
are as follows: 

Broiling 450° to F. . 

coasting (meat) 300^ to 350° F. 

Frying, deep-fat or pan 375° F. 

Exercises (203): 

Indicate whether the following statements are true 
or false by placing a T or an F in the blank: 

1. Liquid is not added when cookii^g with dry 

heat. 

2. Roasting is done in a. broiler. 

3. Deep-fat frying is cooking food by partially 

covering it with hot fat. 

4. The recommended temperatures for roast- 
ing most meats are 300^ to 350^ F. 

204. Identify statements ronc^'ming moist-heat 
cooking as either tme or false. 

MoistrHeat . Cooking Moist heat cooking is the 
process of cooking the food in a liquid, usually^ 
water. The moist -heat cookJng methods described 
in the following paragraphs aie generally used for^ 
Ic^ tfcnder cats of meat and for most vegetables/ 

Simniering is cooking in liquid that is held j w-- 
below the boiling point. When the temperature is 
right for simmering,4he bubbles of steam that rise 
from the bottom of the container disappear before 
reaching the surface of the water. 

Braising is similar to simmering. The food is first 
. browned in a small amount of fat and is then 
simmered in liquid until done. Braising provides 
long, slow cooking and is recommended for the less 
tender cuts of meat. 
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Appendix C 

Mcxiular Behavorial Objective Format 
(Materiel Facilities Course) 



SECTION B, Receiving Procedures 

Hundreds of items arrive at the receiving docks daily. Many of 
these items are urgently needed by our customers while others are. 
needed to replace stock that has already been issued,, THE POINT 
IS— we need to get those items processed and into the hands of our 
customers or in their proper storage locati6n as soon as possible. 



Thjt makes sense, now just 
nhat do I need to know to 
get that Job done? 





Kfy book says that we need to 
achieve these objectives. 



1 



6, 



Upon receipt of suppiies, Ttate tbe required entries 
on the ref^eipt docimient wd identify the coirect 
prccedures for in-cbecidng those itetcs. 
Given a specific receiTisg situation, select the cor- 
rect type of receipt to process, 
identify the procedures for processii^ a Rev:eipt 
Due-In Card. 

Identify the procedures for preparing an Extract 
Receipt Document. ' 

Identify the procedures for procening a receipt in 
any of the fbllcwing conditions: 

a. Serviceable. 

b. IJnserviceabSe. 

c. Partial. 

d. Excess. 

e. Overage. 
/ Shortage. 

Given specific examples of receipts that have been 
processedi identify TRIC/DIC the output docu- 
ments that may results ^ 



8£ 



/ ? 




What do (he, TEX 
code letters P and Z 
mean? 



The output document 
for an unserviceable 
(reparable) receipt is: 



The output document, 
for an unserviceable 
(condemned) receipt 

is: 



These codes are used to help identify the reasons an item is 
received in an unserviceable condition. The letter P identiHes an 
item that probably received damage while in transit. The letter Z 
identifies items that are received with latent (hidden) defects 
which were picbably c?m.*d by the manufacturer. 

If the coho tion of the receipt is unserviceable (reparahl . the 
remote operator can input the due-in card by processing it through 
the 1068 Remote Card Reader. If the condition is unserviceable 
(condemned), the receipt due-in card and the TRM input should be 
sent to ADFM/PCAM for input. 



In the case of the unserviceable (reparable) receipt, the output 
document is normally DD Form 1348-1, Management Notice 1046 
(unserviceable), TRIC REC, (see fig. 5-18), 

The property and output document should be sent to the unser- 
viceable storage area and held there until disposition insliuctions are 
received. 

If you have processed an unserviceable (condemned) receipt, your 
output product is DD Form 1348-1, transfer to Defense Property 
Disposal Office (DPDO), as shown in figure 5-19. 

This output document can be identified by Document Identifica- 
tion Code (DIC) ASJ or A5K. Upon receipt of the A5J or ASK cut- 
put, forward both the property and the document to the supply in- 
spector. The inspector signs or places his stamp and date in block V 
of the DD Fr-:^ \ 348^ 1 . He then for>vards everything to the Pick-up 
and Deliver) Section for final pr »oessing to DPDO. 
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Let*s review the procedures for processing an unservici 
receipt. 



KNOWLEDGE FEEDBACK 23 



1. Transaction Exception Code P in card column 51 of the receipt 
due- in card indicates that 

a. the receipt has a latent defect. 

b. the receipt has been misidentified. 

c. the receipt was damaged while in transit. 

2. What material condition code is entered in card column 7 of the 
receipt due- in card for an unserviceable receipt? 

a. H. 

b. F. 

c. A. 

3. What is the output document normally received as a result of 
processing a reparable receipt? 

a. Notice to stock document. 

b. Due-out release document. 

c. Management Notice 1046 (unserviceable). 

4. Which of the following codes identifies a transfer to DPDO? 

a. A5J. 

b. SHP. 

c. REC. 

5 . What is the TRIC of the input document for a transfer to DPDO? 
J A5K. 

b. REC. 

c. TRM. 

Check your responses at the back of this volume. 



RECEIPT PROCESSING (overage/shortage) 




Something is bugging me, I know what to 
do if the item received is misidentified, 
damaged, or not in a sealed container 
But what do I do if the quantity doesn*t 
agree with the receipt document? 



84 



Appendix D 
High Impact Graphic Text 
(Food Service Course) 

"Why are those coliurj;s blank?" asked -y-^ 
Sabrina . 

"They get filled in later," said Larry. 
"Joe, could you explain this? I've got 
to finish updating all these 14 7s . " 

Joe nodded. "Would you like the full 
srory on the Senior Cook's Requisition?" 
he asked Sabrina. "It's one of our most 
important forms . " 

"Why?" 



"Two reasons. First, you must have it properly filled out to get 
the food for the next meal. And second, it's the ONLY basis for 
figuring the monthly food costs--and you can gu^ss what budgeting 
would be like if you didn't know those costs." 

"Pure chaos." Sabrina shuddered. "All right, tell me about it." 

"Well, the senior cook prepares the first five col-omns — A through 
E-- and the storeroom man fills in the last two — F and G — after 
the senior cook closes out the form. I won't talk about F and G, 
because the senior cook doesn't have to do them." 
^ - 

He looked into the desk drawer. "There should be a copy of that 
chart in here--ah, here it is. It's another one of our training 
aids." 



IS THE SENI<^R COOK'S REQUISITION GIVING YOU GREY HAIRS? 

STOP V;ORRYING-- 

HERE'S WHAT YOU NEED TO KNOW: 

You deal with AF Form 148, Senior Cook's Requisition, in 
two phases: 

PHASE 1. Before the meal, when you use it to take food 
from the storeroom. This involves columns 
B, and C. 

PHASE 2. After the meal, when you account for the food 
you took out. This involves columns D and E. 

PHASE 1. DRAWING YOUR SUPPLIES 



Though no money changes hands, this is sometimes called "purchasing" 
or "buying." Vou will need: 
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W?i^^^n you fir.if^h this chapter^ you should be able to: 



Select fron situations involving the two ti^pes of sanita- 
tion inspections, those situations in which the conditions 
inspected are satisfactory . 

Select from situations involving personal cleanliness and 
health standards, those situations in which the standards 
are met. 



"Lt Silver wants somebody to write some- 
thing on sanitation inspections and personal 
standards," said Sgt Fivest,. ipe, setting dov;n 
his coffee cup. "she wants to be sure every- 
body knows the information — especially since 
we're trying to cat the Hennessy Trophy." 
"Hennessy Trophy, " said Ellen Smart thought- 
fully, thinking of Dave Fastburner's Fro^, 
H:.mer to Hennessy article, 

"Do I hear any volunteers?" asked Five- 
stripe.'' "1*11 do it," said Ellen Smart and Nell Newcome at 
the same time. 

"I need to review the information anyway," added Neli, 
"Since I'm working towards my 5 level, and shift leaders h^iivs 
to know all that. 

"Why don't you do it together?" said Fivestripe. "Do 
whatever you want with it — an illustrated booklet, maybe." 

Ellen and Nell looked at each other. 

"Between yoi>, me, and the duplicating machine," sard Nell, 
"I think we could . ^' 

. ."Be sure to include both the medical inspection and the 
supervisor's inspection," said Fivestripe. "And remember, 
these inspections include facilities^ operations, and per- 
sonnel." 

"Right," said Nell. "We'll conclude with personal clean- 
liness and health standards. Next week you'll have the book- 
let." 
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ACE YOU OW DUTY 
TODAY IP SO 

\ 




■a >^ 

.TART WilH \A 
YOU-? y^VD^A 




THAT §GT MARVEL- 
HOW DOES ANYBODY 

PA^S HEC INSPECTION 



BY MAINTAINING THE STAN- 

/"vU OPERATIONS, AND PEP- 
A* -M PROPER r 

CONDITION, THEY ( 
' -^-^ OAN PASS ANY J( 



A GOOD FOOD SERVICE 

SPECIALIST TA*:rS PRIDE IN 

HIS WORiC, SO HE 

MAINTAINS PROPER 
STANDARDS OF y 
PERSONAL HY- > )' 
GIENE ALL THE 

TIME / 




I ALWAYS DID WASH MY HANDS 
EVERY DAY BUT kU. THESE NEW 
RULES... 



rHEYeE IMPORTANT/ GOOD 



,f^4PERS0NAL HY61ENE DOES 




/f-'ATwo THINGS. n-Haps/|^.>j 

k TV^EEP THE COOK. ' " 



'^^^EALTHX AND 1%./ 
!T HELPS PREVENT THE ^ 
6P3EAD OF FOOD PO'SONING ORfiANlSMS\ ^ 



fUKE STAPHYLOCOCCUS, 
CLOSTRIDIUhA, A.ND 




HESE STANDARDS OF PERSONAL 
;ALTH AND CLEANLINESS F4LL INTO 
iTHIi^EE GROUPS: GENERAL HEALTH, 
CLEANLINESS AND APPEARANCE, 
SANITARY MANNERS, AND \MOPX\UQ 
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Appendix E 



INTRODUCTORY INFORMATION 

DO NOT USE THE MACHINE- SCOR E D ANSWER SHEET FOR THIS PAGE 

Please fill in the following information in the blanks that are 
provided or circle the appropriate letter: 

*%« 

1. How long have you been in the service? / years, npnths . 



2. How old are you? years. 

3. What is your ethnic affiliation? 

a. Caucasian 

b. Oriental 

c. Black 

d. Spanish Surname 

e. American Indian (including Aleuts and Eskimo) 

f . ' /ther , please specify 



4. Indicate your present educational level from the four choices 
below. 

» . Non-high school graduate 

b. GED - non-high school graduate 

c. High school graduate 

d. Teclinical school graduate (not VSh.^ ) 

5. Indicate your present educational level from the five choices 
below. 

a. No college work 

b. Some undergraduate work a allege level 

c. Associate degree 

d. Bachelors degree 

e. Post-graduate degree 

6. Please indicate your sex: 

a. Male 

b. Female 

7. Are you married? 

a. Yes 

b. Wo 

8. How many dependents do you have (including bpouse)? 

a. None 

b. One 

c. Two 

d. Three or more 

PROCEED TO PART I ON NEXT PAGE 
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2CI COURSE RECEIPT CARD 

(Enter date of receipt 

I recei /ed nr/ ECI Course package on: 


Print Nam* (iaat, initials) 


SSAN 


l^is card is being sent to a cros» section 
of students to detenrine the average 
number of days needed to coi^lete ECI 
courses. It vill 2lLso provide us with 
mail transit time information. The code 
nujrfcer below identifies your enrollinent 
for statisticaa purposes only. The 
iaformation on this card has no other 
bearing on your course participation. 
Please return immediately. 


Privacy Act Statement 

1. Authority: 44 USC 3101 

2. Principle Purpose: 

To determine date of course 
package receipt by ECI 
students. 

3. Routine Uses: 
Statistical purposes. 
Students name and SSAN will 
not be used in any formal 
research report. 

4. Disclosure: Voluntary. 
Name and SSAN are requested 
solely to tasise matching 
course receipt data with 
actual enrollr-mt dates. 

May 1976 


llment Code 



r 



ECI 

GUNTER AFS, AL 36118 



OFFICIAL BUSINESS 
fENAUTY FOa PRIVATE USE* S300 



ECI/EDV 

GUNTfcIt AFS AL 36118 
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Appendix G 

NOTICE 

TO 

OJT MONITOR 

The student to whom this course package is addressed Is 
participating In an evaluation of ECI course materials. This 
evaluation requires that we gather certain Information from 
students as they progress through the course^ We are trying to 
determine how long (in hours and minutes) that he or she spends 
working on these materials. The attached forms provide a 
convenient method for keeping a record of this Information. Feel 
free to duplicate these forms locally If there Is Insufficient 
room on those provided. Please ask the student to jot down the 
date and time spent during each study session. We also ask that 
the student be as accurate as possible as the success of this 
evaluation depends on his cooperation. 

The data obtained will be used for statistical purposes only 
and will have no other bearing on the individual's participatiou 
in this course. Although we have asked the student to enter his 
name on the attached forms, this Is for record keeping purposes 
only. The student will in no way be identified in any final 
report regarding this evaluation. 

There may be other versions of this course on your base, 
perhaps in the WAPS library. It is imperative that * th Student 
DOES "NOT use ANY OTHER course materials other than those 
contained in this course package. 

The student will be asked to turn in these completed time 
forros at the time he takes his course examination. Tl nk you for 
yojr cooperation. 
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Appendix H 

Demographics by Course Treatment 
(Materiel Facilities) 



80 



Raco ; 



ronvent icnal 



White 52 

Black 17 

Oriental 3 

Spanish 1 

Indian 0 

Other 1 
Missing data 1 

Totals 75 



3 OF 



51 
17 
1 
1 
1 
3 
1 

75 



Modular 



±o 
3 
5 
0 
i 
0 

83 



lotai 



161 
50 
7 
7 

5 
2 

233 



Sex ; 



Male 58 

Female 16 

Missing data 1 

Totals 75 



U9 
25 
1 

75 



6U 
19 
0 

83 



171 
60 
2 

233 



Education: 

Did not finish 
high school 15 
High School 
Graduate U3 
Some college 13 
Associate 
Degree 1 
Bachelor' s 
degree ^ 
Post Graduate 
Work 1 
Missing data 1 



Totals 



75 



13 

U5 
15 

0 

1 

0 
1 

75 



16 

U6 
16 

3 

1 

0 
1 

83 



13U 

U 

3 

1 
3 

2j3 



Marital status: 

Married 16 

Single 58 

Missing data 1 

Totals 75 



21 
53 
1 

75 



21 
60 
2 

83 



58 
171 
U 

233 



Dependents : 
Yes 
No 



17 
58 



19 
56 



23 
60 



59 
174 



Totals 



75 



83 



233 



31 



5: 



Appendix I 



Demographics by Category and Treatment e 
(Food Services'^ 



oroup 
Category 



Bor 



Resi.: ent 
Population 



Non-Resident 
Population 



Resident 
PoDulatior 



Non-Resident 
Population 



Total 



Race: 



Whi-e 

Black 

Oriental 

Spanish 

Inaian 

Other 

Missing Data 



J 
0 
-0 
0 

0 



11 

u 

0 

0 

2 
0 



33 
6 
0 
0 
3 
1 
1 



20 
7 
0 
0 
0 
0 
0 



102 
20 
0 
0 
u 

3 
2 



Totals 



U2 



18 



27 



1"^1 



Sex: 



Males 33 
Females 9 
Missing Data 9 



:a 

0 
0 



3h 

10 

0 



20 
7 

0 



105 
26 
0 



rotals 



U2 



13 



44 



27 



131 



Education : 

Did not finish 
High School 1 
High School 
Graduate 32 
Some College 8 
Assoc. Degree 1 
Bach. Degree 0 
Post Graduate 
Work 0 
Missing Data 0 



11 
4 
1 
1 

1 

0 



29 
10 
1 
0 

0 
4 



20 
5 
0 
1 

0 
0 



92 
27 
3 
2 

1 
4 



Totals 



42 



18 



44 



27 



131 



Marital Status: 

Mnrried 5 

Single 37 

Missing Data 0 



8 

10 
0 



3 
36 
5 



14 
13 
0 



30 
96 
5 



Tctals 



U2 



18 



44 



27 



131 



Dependents : 

Yes 4 

No 38 

Missing Data 0 



0 
1 
17 



6 
37 
1 



1 
0 
26 



11 
76 
44 



Totals 
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42 



18 



44 



27 



131 



The subject matter in each of the following paragraphs Is directly 
related to your Air Force specialty. Selected words \iere left out' 
of the paragraphs, and blank spaces were placed where the words were 
removed. Your task will be to fill in the blank spaces with the word 
which you feel best completes the meaning of the sentence. Follow 
thct' simple steps: 

1. PRINT only ONE word in each blank. 

2. Try to fill in every blank. Don't be afraid to guess. 

Z. You may skip the hard blanks and come back tr them when you have 

finished the easier ones. 
4. PRINT neatly. 

After you have filled in each blank with the word you think would best 
fit, check your paper for completeness and turn this booklet (with 
student survey answer sheet) in to your test administrator. 

SHIPMENTS 

The directed shipment is the result of base supply receiving shipping 
orders or directives from agencies such as Air Force Logistics Command, 
General Services Administration, or Defense Supply Agency. The 
dzrected shipment can be identified Document 



Identifier Codes A2x and A4x. Nondlrected shipments are 

those that result from local management decisions. This 

type of shiimcyit is idpntified by Transaction identifioation 

Code SHP. 

STORAGE FACILITIES 

A general -purpose warehouse is used for the storage of many kinds of items. 
It Is constructed with a roof, side walls , and end walls. 



This type of facility normally has both truck loading 

platforms or docks and rail car loading platform . These 

platforms provide easy access to the receiving and storage 

areas of base supply. As a general rule, an item 

that does not require special storage or handling i" stored 
in the general -purpose war*ehouse 
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INSPECTION 

Any Item not received in a sealed container should be sent to the 

supply Inspection. The __ "f is true 1f the '■'^forrr<^^io'^ on 

the condition tag not agree with the '"^^^"-P^ 

document, or If the thinks l e receipt itself '^-l' 



be damaged. Upon receir-t -" such a^. item, the ^^-P'? 



inspector Is responsible for i'?te2^^n7:>v ^he condition, status, and 



identiry the item. 



IN-CHECKING PROCEDURES 
All receipt processing begins with off-loading of supDlies. While 

o ff- Lauding ^ ^he 1n-checker has two ^^^y important tasks to 

consider. ^^^'-^ , the supplies must be segregated by 

priority. The high -Mo^-ity receipts (priority 01 through 

) are placed in one ^^-'^ , and the routine 

receipts ( P^'io^'^h 09 through 20) are in another. 

EXTRACT RECEIPT jQCUMENT 
If the extract document is required because of split receipt, part service- 
able and part unserviceable, ther« be an extra step 

the preparation. You must -^^^^^ change the 

quantity on original receipt to reflect ^"^^ . 



the portion that is . no other changes are required 

When you prepare the extract copy for the unserviceable portion 

enter condition code F ^'^ card column 7 and reflect 

only the unserviceable amount in card columns 25 through 29. 
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Semantic Differential Attitude Measure 

YOUR mim ABOUT 1}£ ECI TL IOOIG USED IN THIS 



^ • 

Each ; the-J«teen .items on the next page has two contrasting 
adject ves. For each item, please indicate on the sewn 
point scale your feelings about the textbooks ybu studied as 
described by the adjectives. For example; For item (1) , if 
ynu presently feel that the textbooks were - 

Extremely Good Circle 1 

Very Good Circle 2 

<^ood .Circle 3 

Neither Good nor Bad Circle 4 

B«a Circle 5 

Very fiad Circle 6 

Extremely Bad Circle 7 

For item (2) , if you presently feel that the textbooks 4fere - 

Ext-emely Dull Circle 1 

Dull. .Circle 2 

D^All Circle 3 

Neither Dull nor Interesting Circle 4 

Ir+:f*resting Circle 5 

Ver^^ Interesting Circle 6 

Extremely Interesting Circle 7 

ANSWER ITEMS 3-1^ IN A SIMILAR MANNER , 

In marking the items, keep in mind that this is a test of ham 
you feel about the textbooks you stxadied. sometimes you aay ^ 
feel as though you've had the same item before on the te»t. 
This will not be the case^ so do not look back and forth 
through the items. Do iiot try to remember how you checked 
similar items earlier tlie test. Make each item a separate 
and' independent judgement/ WorH at a fairly high speed 
through this test. Do not wrry or puzzle over individual 
items. It is your first impression, tfoe inediate feelings" 
about the items that we want. On the other hand, please do 
M% be careless, bec%ui.e we want your true impressions. 
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I m THftT MY Fri Tmnv fm^^ .; yppp. 

1. Good ^ 

1 2 3 4 5 6 7 

2. Dull . Intereatinc 

1 2 3 4 5 6 7 

2. '-aat Slow 

i 2 3 4 5 6 7 

4. Tonse Relaxed 

12 3 4^67 

5. Unfair ; rair 

12 3 4 5 6 7 

6. Deep , Shallow 

1 2 3 4 5 6 7 

7. Valuable Vtorthless 

1 2 3 4 5 6 7 

0. Active Passive 

1 2 3 4 5 6 .7 

9. Easy „ oirficuit 

12 3 4 5 6 7 

10. Inflexble Flexible 

1 2 3 4 5 6 7 

11. Excituig Boring 

12 3 '4 5 6 7 

12. Unpleasant _ P^pasant 

1 2 3 4 5 .6 7 

13. Depressing — Encouraging 

1 2 3 4 5 6 7 

14. Safe Dangerout 

1 2 3 4 5 . e' -» 

15. Weak . strong 

1 2 3 4 5 6 7 



PROCEED TO PART 3 ON NEXT PAGE. 
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Stratification 


of Subjects with 


Resident School 


Preparatii 




U±OZG 


\ Scores, Sex, and 


Treatment Assignment 






BOF 


HIG 




Cic^e 


rotai 


Male Female 


Male 


Female 


Scores 


Freq 


Freq Freq 


Freq 


Freq 


7 
8 


1 
1 


1 


1 




9 


1 1 

H 


1 1 


2 




10 




1 


2 




11 




2 




1 


12 


1 




1 




13 










lU 


O 


2 


1 ■ 




15 


b 


2 


2 


1 


16 


b 


2 1 


2 


1 


1 / 




1 






18 


1 1 
H 


3 


1 


■ 


±y 


li 


1 


2 


1 


on 




1 1 


3 




Zl 


D 


2 1 


2 




ZZ 


Q 
O 


3 1 


3 


1 






1 2 


1 






O 


1 


2 




25 


12 


3 2 


6 


1 


26 


6 


3 


2 


1 


27 


10 


3 2 


3 


2 


28 


6 


2 1 


2 


1 


29 


1 




1 




30 




1 1 


2 




31 




1 1 


1 


1 


32 


1 


1 






33 










3U 


1 




1 


t 


35 










36 


1 


1 
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Appendix M 

Classification of Survey Items into 
' Types 'and Subtypes 

The twenty-five items from the student survey were classified 
into three types: (1) those pertaining to the texts, (2) those per- 
taining to the course examination, and (3) those pertaining to the 
stature of ECI, The first two were further divided into three sub- 
types pertaining to difficulty level, coverage of objectives, and job 
relevance. The items occurring in each category are shown below. 



TABLE 11 
CLASSIFICATION OF SURVEY ITEMS 





ITEM SUBTYPES 


Difficulty 
Level 


Objective 
Coverage 


Job 
Relevance 


ITEM 
TYPES 


(1) Tfext 

Materials 


1, 5, 6, 7 


2, 3, U 


iq^, 12, 13 


(2) ' Course 

Examination 


21, 22 


19, 23, 25 


20, 24 


(3) Stature of Extension Course Institute Item- " and 18, 



/ — 



9S 



Appendix N 

The Survey Items and Pertinent Response Data 
Th^ results of the survey are tabulated below, and each item and 
iu^ alternatives are stated as they occurred on the survey. Additionally, 
beneath the last alternative, a combination of alternatives is stipu- 
lated when needed (e.g., A+B indicates the number of students responding 
to alternatives A and B), This is followed by a row showing the number 
of students responding to each J.tem. To the right of each alternative, 
the percentage of students responding is reported for the. three materiel 
facilities and two food service treatment groups. Applicable percentages 
for the combined alternativjes are shown in a similar manner. The last 
row- of each item shows the number of stu<ients responding to t^at item. 
The first number in parentheses represents the total number of materiel 
facilities students who responded to the item and the second number 
represents total responses for food service students. The next five 
numbers represent the T-.uinber of students responding from the various 
treatment groups. 

For each item a decision was made to determine which of the possible , 
alternatives or combination of alternatives represented a nef.-ttive point 
of view. For example, for item 1 students who responded with M or B 
reflected the negative opinion tn^rz their course material:^ w^: e 
difficult for them at thf^v- pyi^esf level of training. 

The percen-'.ar — to the rig;^^ ' M-.y-r^e^smrT-" 

the number <\ iM^.^rxi^I Xc;. . ?^dents from the c )vr ♦^^^ *Fe 

ment group who indicated a n«^ opinion about fhei' ^^^f^LU, 



per nMitage in coluinn 2 represents the number of students who indlcatrni a 
negative opinion about the BOP course inaterials and coluriH 3 represents 
similar information about the modular treatment group. The perr ^^^ntages 
in columns 5 and 6 represenr negative opinions expressed by food service 
students from the BOF and high impact graphic treatment groups respective 
ly. A response or cpCibination of responses that reflected a negative 
opinion was identified for all items on the survey except one Citem^ 18) 
which called for a nonevaluative statemerft about Community College of 
the Air Force transcripting. 



J 
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Item Response Data 



MATERIELS FACILITIES 



FOOD SERVICES 



ITEM 



CONVEN- 
TIONAL 



BOF 



MODULAR BOF 



HIGH ^ 

IMPACT 

GRAPHIC 



1. The material in the text was too difficult for :r4e at •my present 
lev^l of training (Measure: Text difficulty.) 



A, Strongly agree 


^.U9 


9.41 


U.39 


3.50 


2.59 


B. Agree 


10.11 


23.52 


18.68 


12.28 


7.79 


C. Disagree 


77.52 


63.52 


70.32 


84.91 


68.83 


D, Strongly Disagree 


7.86 


3.52 


6.59 


19.29 


20.77 


A+B 


1U.60 


32.93 


23.07 


15.78 


10.38 


(265) (13U) 


89 


85 


91 


57 


77 


2, More illustrations or foldouts 


are needed to support the 


text, (Mea 


sure : 


Text coverage. ) 












A. Strongly agree 


3.37 


13,95 


1U.28 


15.51 


3.89 


B. Agree 


U7.19 


39.53 


29.67 


37.93 


18.18 


C, Disagree 


U3.82 


J+4.18 


50.54 


43.*10 


59.74 


D. ^Strongly disagree 


5.61 


2.32 , 


5.49 


3.44 


18.18 


A+B 


50.56 


53. U8 


43.95 


53.41* 


^22, 07 


(266) ' (135) . 


89 ' 


86 


91 


58 


77 



3. 



% 

The text adequately covers the behavioral objectives. (Measure: Text coverage.) 



A. 


Strongly agree 


1.14 


1.16 


3.26 


12.06 


6-.U1 


B. 


Agree 


65.51 


' 56.97 


58.69 - 


67.24 


69.23 


C. 


Disagree 


27.58 


33.72 


28.26 


20.69 


20.51 


D. ' 


Strongly disagree 




8.14 


. 9.78 ■ 


0;00 


3.84 


C+D 




■ 3. 31 


41.86 


38.04 


20.69 


2U.35 




(265) (135) 


'£17 


36 


92 


58 / 


78 






IBl 








1 



ERIC 



MATERIELS FACILITIES 



ITEM 



cc^r/EJi- 

TIONAL 



30F 



MODULAR BOF 



FOOD SERVICES 
HIGH 
IMPACT 
GRAPHIC 



^. The questions at the end oi' each 'behavioral objective actually 
tested the objective. (Meas'ire: Text coverage.) 





A 

n. 


Strongly agree 


J - ^ J 


U.65 


9.78 


17. 2U 


8.97 




tj 


Agre<* 


i . 8b 


67. 


70.65 


58.62 


73.07 




L« 


Disagree ^ 




25.58 


15.21 


2U.13 


lU.lO 




D. 


Strongly disagree 


2.29 


2,?2 


4.3U 


0.00 


3.8U 




C+D 




17.23 


27.90 


19.55 


2i^.l3 


17. 9U 






^2^) (136) 


87 


86 


92 


58 




5. 


The 


text is easy to r^ad. 


(Measure: 


Text difficulty. ) 








A. 


Strongly agree 


20.65 ^ 


17. 2U 


U1.02 


17. 2U 


U1.02 




B. 


Agree 


53.93 ' 


S9.30 


57.60 


63.79 


U6.15 


K 


C. 


Disagree 


2U.71 


26. 7U 


19.56 


17. 2U 


10. 2^^ 




D. 


Stromgly .;isagreK 


' 7.86 


6.97 


/• 7 T 
^ , ,^ / 


1.72 






C+D 




32.57 


33,71 


21.73: 


18.96 








^126") :^ G) 


89 


' 86 


92 J 


58 




6. 


The 


text :.iontairL^sii:ir ^ many 


technican 


terms. (Me 


aaure; 


Text difficulty. 




A. 


Stror^i^^ly agr^^ 


11.23 


12.79 


io.87 


70. 3U 






B. 


Agre^ 


3U,83* 


36. OU 


31.52 


18.96 








Disaigr^ee 


51,68 


UU.IB 


53.26 


63.79 


70. 




D. 


Strongly 1 ; c^-- 


2.2U^ 


6.97 




5.8§ 






AfB 




♦36.06 


U8.83' ' 


42.39 


29. 5o 


15>..^- 






(267) , ^^6) 


89 ^ 


86 . 


92 


58 


76 



10£' 



ERIC 



ERIC 



MATERIELS FACILITIES FOOD SERVIQPS ^ 

HIGH 

CONVEN- IMPACT 

ITEM TIONAL BOF MODULAR BOF GRAPHIC 

~ 

7. The number of acronyms and abbreviations used (i.e., ECI, USAF, 

DANTES, ETC.) was excessive. , 

. A. Strong^.y Agr*=»3 20. U5 18.82 2 : . 07 10.25 

B. Agree ' ^+0.90 43. 52 3'*. 55 2^.B>b lU.lO 

C. Disagree 36.36 31.76 3:8. ^-7..^ 70.51 

D. Strongly disagree 2.27 5.88 1^ V- -- /.^^^ 
A+B 61.35 62.34 t 43 

(264) (ld6) 88 85 



8. Often techni-cal '^.er-rms or acronyms were us^d that : . uq" been 
defined. (Mearur^^- Tevt difficulty.) 



A. Strongly *~ , 6.74 . 18.82 3.. 6^: 3.8M;- 

B. Ajcree 41.57 58.82 1*^3.4" 2 3.6"" 26.92 
■ C. La.?2*-agrerr ^ 47.19 41.17 iMi,5(.. 7 5.J:- t) n6.66 

D.. ;:^-t:r-n;ilv * 4.49 I.IT 3.26^ 0.^.0 2.56 

A-*-H^ 48.31 ,57.6^ 52.16 24, . 30'. 86 

(2^^.; (136) 89 85 92 58 78'. 

9. rrffj/.r was ^ , o understand. (Measure: Text difficulty.) 

A :Strong-A 5.61 3.52 9.78 13. 9 26.92 

^g3:^ee . 57.30 52.9^ 53.26 57. 61.53 

^ Disagree 33.70 35.29 31.52 15. fa 8.97 

i St-rongly disagree 3.37 8.23 5.43 3.^4 2.56 

Cn^ 37.07 43.52 36.95 18.95 XI. 52 

(266) (136) 89 85 92 58 78* 









MATERIE 


L5 FACI 


LITIES 


FOOD 


SERVICES 


















HIGH 


















IMPACT 


ITEM 






■ iO: 




dOF 






GRAE'irtHC 


10. 


The illustration.s and foldo 


r..;ts 






T supported thtie t. 


xt, (Measiiire: 


Text 


coverage, ) 


















A« Strongly agree 




3. 


J7 


U52 


14,: 


10, 3U 


10. 2S. 




Agree 




56 . 


18 


'8,82 


51. 


51.72 


66«6C- 




C, Disagree 




3fi 




J4,ll 


' 19. 


31.03 


2C .. 5l^' 




Strongly disagree 








3/52 




6,89 


J. 56. 




C+D 








7,63 


23..^C 


37,92 


. „ 




(266) - (136) 




6-: 




: 5 


92 


58 




11. 


Do you think that your ECI 


cou 






ycu too 


' much ma L^'i 


iL, more 


than yciu 


needed at your particular stage 


in 






'orce? 


(Measure:: 


\ ^2Xt relevance - ) 


• 


A. Yes 






96' 


53,33 


36,^5 


2U,13 


18. 8F 




B. No 




71. 


9L 


61+. 28 


61,^^5 


75,85 


80.22 




A 

(265) (135) 










92 


58 


77 


12. 


There ,is iDore material in thi 




xt rha 


;r, needed to do my 


ob, (Measure : 


Text 


relevance . ) 


















StrongJ.y agree 




16. 


85 


26,19 


31. 52 


16.07 


lU.lO 




B • Agr eie 




3^4,83 


30.95 


32.60 


30.25 


• 28,20 




C. Disagree 








39,28 


28.26 


50.00 


47. 43 




D, Strongly - disagree 




1. 


96 ' 


3.57 


7,6C 


3.57 


10.25 




A+B. 




51,68 


57.14 


6U.12 


U6,U2 


42.30 




(265) (13U) 




89 




8U - 


92 


56 


78 



.A 



104 



ITEM 



MATERIELS FACILITIES 

COWEN- 
TIONAL EOF 



FOOD SERVICES 
HIGH 
IMPACT 
3F GRAPHIC 



13. What percentage of topics discussed in the text do not relate to your 
present job? (Measure: Text relevance. ) 



A- 


75 percent 




21. 


34 


23. 


52 


20. 


87 


21. 


42 


14. 


28 


B. 


50 percent 




26. 


96 


30. 


58 


28. 


57 


25. 


00 


23. 


37 


C. 


25 percent 




35. 


95 


35. 


29 


39. 


56 


32. 


14 


36. 


36 


D. 


All topics 


relate to 
























my present 


job 


15. 


73 


10. 


58 


10 


98 


21. 


42 


25. 


97 


A+B 






48. 


30 




10 


49. 


44 


46. 


42 


37, 


65 




(265) 


(133) 


89 




85 




91 




56 




77 





14. The tasks described in this course are more complex than the 
tasks I perform on the job. (Measure: Text relevance.) 





A, Stijongly agree 


14, 60 


24.70 


28,26 


8,62 


8,97 




B. Agree 


48,31 


42,35 


36,95 


32,75 


32,05 




C. Disagree 


34,83 


32.94 


31,52 


51,72 


52,56 




D. Strongly disagree 


2,34 


0,00 


3.26 


6,89 


6.41 




AfB 


62,91 


67,05 


65,21 


41.37 


41,02 




(266) (136) 


89 


•85 


92 , 


58 


78 


15. 


Of the different 'types of 


equipment discussed 


in thif; course, I have 




used 


or been in contact with 


(Measure: 


Text 


releva.^ce, ) 








A. all of them. 


10,11 


15,11 


14,13 


22.41 


21,79 




B. most of them* 


51,68 


41. 36 


41.30 


62.06 


53.84 




C, few of them. 


19,10 


22,09 


23.91 


13.79 


15.38 


V 


D, very few of the^p. 


19,10 


20,93 


20.65 


1,72 


8,97 




C+D 


. 38.20 


43,02 


44, 56 


15. SI 


24,35 




(267) (13^) 


89 


86 


92 


58 


78 



105 



ERIC 



KATERIELS FACILITIES 



ITEM 



CONVEN- 
TIONAL 



EOF 



MODULAR 



FOOD SERVICES 

HIGH 
IMPACT 

EOF GRAPHIC 



16. Of the tasks described in this course, I have performed or come 
in contact with (Measure: Text relevance.) 



A. all of them. D.OO 0.00 2.17 10. 3U 10.25 

B. most of them. 39.32 29.41 36.95 72.^1 57.69 

C. few of them. 45.06 56. U7 i+3.U7 13.79 28.20 

D. very few of them. 1U.60 1^4.11 17.39 3.UU 3.8U 
C+D 60.66 70.58 60.86 17.23 32. 0^4 

(266) (136) 89 85 92 58 78 



17. Does it give you more confidence in the quality of your ECI course 

to know that ECI has been nationally accr«dited by the National Home Study - 

Council? (Measure: ECI st;dture. ) 



A. 


Yes 


31.46 


22. 


35 


29. 


67 


39. 


65 


38. 


U6 


B. 


No 


37.07 


uo. 


00 


29. 


67 


25. 


86 


29. 


U8 


C. 


Accreditation has no 


relation 


















B 


to course quality 


29.21 


37. 


6U 


' UO. 


65 


32. 


75 


30. 


76 




(265) (136) 


89 


85 




91 




58 




78 





18. Soon, many ECI courses will be transcripted Ly the Community College 
of the Air Force. What will be the effect of lifting ECI course completions 
on your transcript? (Measure: £21 stature.) 

A. It shows that the civilian 
community recognizes the 

training I have received. 5.68 13.09 10.11 20,69 14.47 

B. It will help me secure 
employment when I leave 

the service. 6.81 11.90 6.74 8.52 10.52 



106 



ITEM 



MATERIELS FACILITIES 



CONVEN- 
TIONAL 



BOF 



MODULAR BOF 



FOOD SERVICES 
. HIGH 
IMPACT 
GRAPHIC 



D. 



It will help reflect 
all of my Air Force 

training. 13.63 13.09 1U.60 6.89 9.21 

All of the above. 73.86 61.90 68.52 63.79 65.78 

(261) (13U) 88 8U 89 58 76 



19. The course examination asks questions about material not in the 
text. ' (Measure: Examination coverage.) 



job. 



A. 


Strongly agree 


7.95 


20.00 


10.87 


6.78 


2.56 


B. 


Agree 


26.13 


3U.11 


29.34 


20.33 


19.23 


C. 


Disagree 


5U.5U 


42.35 


51.08 


59.32 


65.38 


D. 


Strongly disagree 


11.36 


3.52 


8.69 


13.55 


12.82 


A+B 




3U.08 


54.11 


40.21 


27.11 


21.79 




(265) . (137) 


88 


85 


92 


59 


78 


The 


course examination covered the most 


important 


concepts relating 


to my 


( Measure : Examination 


relevance. ) 










A. 


Stixjngly agree 


13.63 


3. 48 


7.60 


10.16 


2. '39 


B. 


Agree 


ij^6.59 


51.16 ' 


42.39 


64.40 


76.62 


C. 


Disagree 


31.81 


34.88 


29.34 


22.03 


11.68 


D. 


Strongly disagree 


7.95 


10.46 


20.65 


3.39 


9.09 


C-l-D 




39.76 


45.34 


49.99 


25.42 


20.77 




(266) .(136) 


88 


86 


92 


59 


77 



. 10"! 



MATERIELS FACILITIES 



ITEM 

22. The couz 
the material. 



corrv^EN- 

TICNAL 



examination is toe easy to really tes^t 
(Measure: Exar^ination difficulty.) 



FOCD SERVICES 
HIGH 
IMPACT 
GRAPHIC 



3QF 



knowledge of 



A. 


Strongly agree 


o.oc 


0.00 


2. 


22 


8.47 


1.29 


B. 


Agree 


6.74 


6.97 


1. 


11 


8.47 


14.28 


n 


Disagree 


79.77 


68.60 


80. 


00 


77.96 


74.02 


D. 


Strongly disagree 


13. U8 


24.41 


16. 


66 


5.08 


10.39 


A+B 




6.74 


6.97 


3. 


33 


16.94 


15.57 




(265) (136) 


89 


86 


90 




59 


77 


23. Most course examination 


questions came directly 


from 


the 


volume 




review exercises. (Measure: 


Examination 


coverage. ) 










A. 


Strongly agree 


19.31 


4.70 


5. 


43 


8.62 


14.47 


B. 


Agree 


59.09 


44,70 


35. 


87 


67.24 


57.89 


C. 


Disagree 


18.18 


31.7«S 


38. 


04 


22.41 


25.00 


D. 


Strongly disag- 


3.40 


18.82 


20. 


6':> 


1.72 


2.63 


C+D 




21.58 


50.58 


58. 


69 


24.13 


27.63 




(265) (134) 


88 


85 


92 




58 


76 




■ course examination 


questions are 


about subjects 


important to my 


job. 




Examination relevance.) 












A 


itrongly agree 


9.09 


4.65^ 


7. 


69 


6.89 


16.88 


B. 


Agree 


48.86 


50.00 


48. 


35 


70.69 


68.83 


C. 


Disagree 


29.54 


32.5-^ 


30. 


76 


22.41 


11.68 


D. 


Strongly disagree 


12.50 


12.79 


13. 


18 


0.00 


2.59 


C+D 




42,04 


45.34 


43. 


94 


22.41 


14.27 




(265) (135) 


88 


86 


91 




58 


77 






lot 
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MATERIELS FACILITIES 



ITEM 



CONVEN- 
TIONAL 



BOF 



MODULAR 



25.0 When preparing for the courSf^ examinations, I studied: , 
Examination coverage. ) 



FOOD SERVICES 
HIGH 
IMPACT 
BOF GRAPHIC 



(Measure: 



B. 



C. 
D. 



Only the volume 

review exercises 

The text and the 

volume review 

exercises 

Only the text 

The text, the volume 

review exercises, and 

the chapter review 

exercises 



A+C 



(259) 



(132) 



29.88 



22.98 



US. 67 
33.32 
87 



20.00 



37.64 
3.52 



38.82 
23.52 
85 



22'. 98 



29.88 
2.29 



UU.82 
25.27 
87 



22.80 



36.84 
7.01 



33.33 
29.81 
57 



20.00 



28.00 
5.33 



46.66 
25.33 
75 
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Appendix 0 

The Frequency of Occurrance of Arrays 
Beth the Material Facilities and the Food Service study had groups N 
that were issued innovative textbooks. For the Materiel Facilities 
study, this was the modular group and for the Food §ervice study^T^he 
high impact graphics gpoup. As the modular and the high impact graphics 
groups were the focus of attention, analysis of the survey items high- 
lighting these groups was a primary concern. To facilitate this analy- 
sis the survey items were first grouped according to their type and 
subtypes, then the relative standing of the frequency of response for 
each treatment group was examined within each oi- the seven groupings. 

For ease of descrij^ion, the relative standings on the frequency of 
responses of or each item was represented by an array. For example, 
letting C, B, and M respectively represent the conventional, BOF, and 
modular courses for the materiel faViiities course, the array CBM would 
indicate that the conventional course had the lowest percentage of 
subjects holding a derogatory opinion about the item, the BOF the next 
lowest and the modular th| highest. Letting B and H respectively re- 
present the BOF and HIG courses for the food service course, the arrcty 
BH would indicate that the BOF course had ,the lowest percentage of 
subjects holding a derogatory opijriion about the item and the high impact 
graphics course the highest, ^ 

Using the above definition of an^ array, it is noted that there are 
six possible arrays for the materiel facilities coiirse and two for 



110 



100 

the food service course. Possible arrays for the materiel facilities 
courses were CMB, MCB, CBM, .MBC, BMC, and BCM. Possible arrays for 
K the food service courses were BH and HB. 

/ Table 11 shows the frequency of occurrance for arrays and rankings. 

Column 1 identifies the array and column 2 indicates the frequency of 
the arrays occurrance. Column 3 identifies the specific treatment and 
columns 4, 5, and 6 identifies the treatments ranking. For example of 
the 24 items on the survey' that were considered, the most frequently 
occurring array was CM^ which occurs on 10 items. Then too, treatment 
C was ranked first (most positive) 16 times^ 2nd (less positive) 6 
times, and ,3rd (most negative) 2 times. Frequency of occurrancy of arrays 
and treatment rankings are 'likewise presented for the food service where 
only two rankings prevailed. The arrays are described above are also use^^ 
in Tables 12 and 13 where separate depcitions are given for the materiel 
. facilities course and the food service courses, re^ectively. Tables 
12 and 13 group the items into seven categories depending on the intent 
of the item. * ' 
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TABLE 12 



FREQUENCY OF OCCURRANCE OF ARRAYS 
AND TREATMENT RANKINGS 



Possible 



Frequency of Arrays 



Materiel Facilities Courses 



Frequency of F 'Ki,, j Arrays 
Treatment 1st id 3rd 



CMB 

MC3 
CBH 
MBC 
BMC 
BCM 



10 
6 
6 
1 

1 
0 



c 

B 
M 



16 
1 

7 



2 
16 

6 



Food Service Courses 



(IB 
BH 



17 
7 



H 
B 



17 
7 



7 
17 



112 



102 



TABLE 13 



FREQUBNCY OF CURRANCE OF ARRAYS AND TREATMENT 
RAN rNC ^OR SEVEN CATEGORIES 



irequency 



. Frequency of Ranking 
let 



Items 


Text Materials 








•<r' 






1.5,6, 


1. 


Difficulty 


cmb' 


3 


c 


3 


3 


0 


7,8,9 




level 


MCE 


3 


B 


0 


0 


6 












M 


3 


3 


0 


2,3,4, 


2. 


Course 


CMB 


2 


c 


2 


1 


1 


10 




coverage 


MBC 


1. 


B 


0 


1 


3 








MCB 




M 


2 


2 


0 


11,12,13 


3. 


Relationship V 












1«+,15,16 




to job 


CBM 


3 


C 


6 


0 


0 








CMB 


3 


B 


0 


3 


3 












M 


0 


3 


3 



21, 22 



19, 2^, 
25 



20, 24 



Coui'se Examinations 

4. Difficulty 

level MCB 
CBM 



Course 
cover?:, 



BMC 
CBM 
CMB 



Relationship 
to job CBM 
CMB 



1 
1 



1 
1 
1 



1 
1 



C 
B 
H 



C 



c 

B 

H 



1 
0 
1 



2 
1 
0 



2 
0 
0 



1 
1 
0 



0 

1 

2 



0 
1 
1 



Items that relate tc^rtatvire or status of ECI course materials 
17, 18 7. Status MCB 1 



C 
B 
H 



0 
0 

1 



1 

0 
0 



0 

1 
1 



1 
1 
1 



0 

1 
1 



0 

1^ 

1.^ 



113 



10 



TABLE 14 



FREQUENCY OF OCCURRANCE OF ARRAYS AND TREATMENT 
RANKINGS FOR SEVEN CATEGORIES 



Frequency Frequency of Ranking 

Type Category Arrays of Arrays Treatment .1st 2nd 



Items 


^ Text Materials 














1,5,6, 
7,8,9 , 


1. Difficulty 
level 


HB 
BH 


S 
1 




H 
B 


5 
1 


1 
5 


2,3,4, 

10 


i/ 

2. / Course 
/ coverage 


HB 
BH 


3 
1 


.- 


H 

1 B 


3 
1 


* 

1 
3 


11,12,13 
14,15,16 


G.- Relationship 

to job HB 
BH 


4 
2 




H 
B 


i 

4 
2 


2 
4 


Course Examination 














21, 22 


4.- Difficulty 
level 


HB 
BH 


1 
i 




H 
B 


1 
1 


1 
1 


19, 23 

25 


5. Course 

coverage 


HB 
BH 


2 
1 


p 


H 
B 


2 
1 


1 

2 


20, 24 


6. Relationship 
^ to job HB 


2 




H 
B 


2 
0 


0 


Items that relate to 


stature or 


status of 


t 

EG I course materials 




17, 18 


7. Status 


BH 


1 




H 
B 


0 

1 


I. 

•"'1 
0 



ERIC 



Appendix P 

:>ntci/]P ()if Materiel Kcillties Students Respofidinc; ^ffstively to Surv^v Items 



2 4 6 8.10 in. 5^) 18 I Z;: 24 26 28 jO 32 34 36 38 40 4? 44 46 i 52 54 56 58 60 i64 66 68 /0 



ITEH 
NO. 



25' 

I? 

9. 



19: 

6 

t 

24^ 
21 



22 N CB 
4 

5 



EX/IMPLF il "f fiodmlar students and It of convinttoittl 
m i ' students perceived item 22 negati»«l 



C ^1 




:i3f 
:ie 

I I I 

ERIC 



I 



I ' ' 

' : I 

! ! ! 

' i I ! 

Ill; 



1 



I ! 



CB 



H 
H 



C B 
MC 

B. |H' ; ; 

c; M ' B 



B 



! ■ I 



I i 



' ! i 



I ! 



ic 



^ f 



I I 



\ M, : t. Bi. 



IB Mi 



I I 



I I 



t 



! I 



j I 



d 1 b; 

•C i fi i IB 



I ■ I 



1" : 

I I 



I 1 
I • ' 



I I 



HCB 

! - ! 
CH 



C I' KB 



Si. 



Percentage of Food Sjrvi ? Stt r.^ *«iinG Necirr --ly to S^ir«^ ' sterns 

^iiw— <I!P— > ■!!€■. I II III n'lifcgliii ■■ 11*1' 



ITtH ! n mo 12 1'4 1'e ^2 24 it 23 :0'32 . ( k 1^1 51? 52 54 Str % i 



nof hfeti ^0ifacr fj-aphic students anj 
li^W' $tylsr perceived 1 ! 



20 
21 
19 

2; 

¥ 

i 

/ 



B I I. 
!23i 



Is! 

i'i. 

i'i 
1? 



... i.... 



I I 



HB 



t I ' 

H, : .! B 
'H i B 



B \ 



:b 



1-! 

I 



^ ! 


j 




..L 
< 1 




1 


1 


4 . 

1 


l-l 

1 

1 ■ 


1 



HB 



1 I 



I ! 



:B 



I I 



! t 



I I 
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Appendix R 
HYPOTHESES 



Materiel FaciliTies 
Specialist Srudy 



Focxl Ser^wicfj Study 
Effects of Instruction on Performance 



: Students who study modular course materials 
Will show significantly better performance scores 
than those students who study BOF materials. 



H^: That students who study high 
impect graphic course nat-arials will 
show significantly better performance 
scores than those who study BOF 
materials . 



Students who study modular courses will show 
significantly better perfo7?mance scores than those 
who study conventional materials. 

H3: Students who study BOF materials will show 
significantly better performance scores than stu- 
dents who study conventional materials. 



Hi^: High ability readers will perform signifi 
cantly better than low ability readers. 



Effects of Reading Ability on Perfortrvance 

H2: That high ability readers will 
perform significantly better than 
low ability readers. 



/Jn ique Effects of Treatments on Reading Ability Levels 



H5: High ability readers will show little 
difference in performance from one treatment tr 
anothar while low ability readers will demonstrate 
poorest performance using conventional texts, inter- 
mediate performance with BOF> and liighest perfor- 
mance using the modular materials. 



H3: That iiigh ability readers will 
show little difference in performance 
from one treatment to another while 
niddle and low ability readers will 
demonstrate best performance using 
high impact graphic course materials 
and poorest performance using 30F 
materials. 



Effects of Tcrmat on Student Attitude 



Hfi: Students who study the modular course will 
display a significantly more positive attitude 
toward their training materials than those who 
receive the BOF version. 



H^; That students/who st the 
high impact graphic cours.. vill dis- 
play a signif icantly* mor« positive 
attitude(Stoward their training 
materials than those who receive a 
BOF course. 



H7; Students who study the modular course will 
display a. signif icantly more positive attitude 
toward their training materials than those who 
receive the conventional course. » 



\ 



H9: Students who study BOF course mat»rials will 
display a ^signif icantly more positive attitude 
toward their course materials than those who re- 
ceive .he conventional course. 



Effect of Format on Completion Tines 



Hg: The completion time for students using 
.modular oatenials will be significantly shorter 
than for students using BOF materials. ^ 



H5: The completion tiiae in days for 
students using high impact graphic 
materials will be significantly shorter 
than for students using BOF materials. 



Hiq: The completion xTme for students using 
modular materials vrill be significantly shorter 
than for students using the conventional materials 

H-^l' The completion tin^ for students using BOF 
materials will be significantly shorter than for 
students using conventional materials. 



HP 
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